CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Ian Cooper [MVP]

November 2007 - Posts

  • Architecting LINQ to SQL applications, part 2

    What is LINQ?

    LINQ stands for Language Integrated Query and is a DSL within C# for querying data. It is implemented by language extensions to C# 3.0. It allows us to concisely express queries against our collections of data. LINQ comes in a variety of flavours: Objects, SQL, and XML which allow us to query different sources of data. LINQ is an internal DSL. The query expressions that we write are declarative statements about what we want, to project, restrict, etc. our data. The compiler maps them into imperative statements: method calls, lambda expressions, initializers etc.  

    LINQ gains the flexibility to target different collection types by allowing query expressions against both IEnumerable, for standardized collections, and IQueryable for others. IQueryable exploits the ability in C# 3.0 to treat code as data. A lambda expression can either be represented as a delegate or as an expression tree. Framework authors can provide implementations of IQueryProvider to interpret expression trees and emit code appropriate to the target. So LINQ to SQL takes expression trees and turns them into SQL statements. 

    But as well as providing implementations of IQueryable and IQueryProvider, LINQ also provide additional services for working with those collections. LINQ to SQL provides us with a Table<T> class that allows us to treat a table in an RDBMS as though it were an in-memory collection of objects of type T, supporting CRUD operations on elements in the collection. The associated DataContext allows us to control the persistence of elements within that collection. It is important to recognize this as when we talk about LINQ we often need to be clear if we are talking about a DSL for querying collections of data or new framework classes for those collections of data. Some of the features under the LINQ brand are specific, whereas the DSL for queries is intended to be generic. 

    Layers

    A layered architecture is one organizing principle for software, though not the only one.Think of the layers within software as looking something like a layer cake – each layer stands upon the foundations of the one below. We allow each layer to know about the services offered by the layer below it, but we do not allow the layer to know about the consumers of its services. A layer is independent of layers above it, but dependent on the layers below it. To manage our dependencies, our layer tends to depend on abstractions in the layer below, not concrete details. If we can, we narrow the surface area of our layer, to a small number of services that encapsulate what our clients know about. That allows us to change many of the details of our layer, as long as the published contract for how you interact with the layer remains constant. These points of connection are called facades.We use layers within software to:
    •  Manage complexity; we only care about what is in the layer.
    • Provide alternative implementations; so that we can meet new requirements
    • Allow re-usability at a coarse-grained level; each layer might have value to other consumers

    A Layered architecture

    There are a number of ways to layer our model. Here we divide it up into presentation, application services, domain, and infrastructure services.  

    The presentation layer is responsible for showing information to the user and allowing the user to send us commands.

    The application service layer co-ordinates the response of the domain layer to a request from the presentation layer. Its responsibilities do not include business logic, instead it has organizational or workflow logic that initiates work within the domain. Its state reflects the progress of tasks not the state of the domain.

    The domain layer is the heart of the application, where the information that the business cares about is held and manipulated and rules are applied.

    The infrastructure layer provides services that allow the other layers to work such as persistence and communication. Both the domain layer and the application services layer may use the infrastructure layer.

    One variation is to decide that the application services and infrastructure services do the same thing, insulate the domain from something that wants to communicate with it, or that it wants to communicate to. This, service layer, is a set of adapters, which our application uses to communicate with ports which are the API for anything external such as UI or DB. An adapter, specific to the port type, converts a request on the port into calls on the domain and translates requests from the domain into calls out of the port. So in an MVC web application, the controller is an adapter, which takes requests from a Web UI port, turns that request from HTML into calls into the domain, and then turns the response back into HTML to send to an outgoing port. Cockburn calls this a hexagonal architecture.

     

     The common thread here is that the domain is isolated from other aspects of the application.

    Next time we will talk about which layers LINQ to SQL fits into, what patterns it implements, and how best to work with it.
     

  • Architecting LINQ to SQL applications, part 1

    Architecture for  LINQ to SQL

    People are beginning to ask questions about how to architect applications that use LINQ. It is important to remember that LINQ is both a set of language features and a set of additions to the framework.

    While there are no best practices for LINQ today, because the technology has not been in use long enough, when we look at some of the LINQ providers, such as LINQ to SQL, we can start to develop our best practices based on the foundations of best practices around existing tools. While it provides new ways of solving some existing problems the patterns that it uses are well understood, even in their C# 3.0 clothing. To draw parallels with existing best practice, to see where it can inform us, we need to understand what roles LINQ plays in our Enterprise Architecture. That way, we can work with LINQ when we use it within our own applications rather than against it.

    • What are some of the questions we hope to answer by looking to existing practices?
    • What is the relationship between LINQ to SQL (and LINQ to Entities for that matter) and the data access object (DAO) or Data Mapper pattern?
    • How do we write dynamic queries with LINQ?
    • How do we write an NTier application , when LINQ is part of our mix?

    We’ll try to address some of these issues, and others. Along the way we will define what LINQ is, what enterprise architecture looks like, what the difference between tiers and layers is, and how we should write a distributed system.

    First we will look at what we mean when we refer to LINQ itself, and then at what we mean by layered architectures, in order to establish some groundwork for what is to come.


     

     

    Posted Nov 29 2007, 11:02 PM by Ian Cooper with 11 comment(s)
    Filed under:
  • Whither DDD?

    Last Saturday was DDD6. I’ll post some follow up material from my session later. However, last night Daniel Moth asked an interesting question: what is the future of DDD?

    DeveloperDeveloperDeveloper

    First, let me clear up any international confusion. The name DDD comes from Steve Ballmer’s famous mantra, but we soon shorted the mouthful to DDD. No intentional collision with Domain Driven Design. DDD is a UK equivalent of a code camp. Last weekend was out sixth (we have two per year, so we have been going at this for three years), kindly hosted by Microsoft (thanks to Clare Burgess for all the support). The major rules we have are:

    • No Microsoft speakers
    • Open submission of sessions
    • Open voting on sessions to form the agenda
    • Free to enter
    • Multi-track
    • On a weekend

    The mantra is by the community, for the community, and one of our key goals has been to encourage the growth of speakers from within the community who can share real-world experience with their peers.

    How did it go?

    We had 320 attendees, and gave 20 sessions across 4 tracks. Because my session (on LINQ, layered architectures, and n-tier) was on first, I had a good chance to wander around. The feedback I got from people attending was very positive. From an alt.net perspective it is interesting to note that we had a good set of submissions on how to improve practices or expand horizons, such as Dave Verwer on Iron Ruby, Paul Lockwood on Cruise Control.NET, Richard Fennell on Scrum, Mike Hadlow on Inversion of Control, Michael Foord on IronPython and Dynamic Languages in general, and Ben Hall on TDD with MbUnit. Gary Short and I both gave patterns and practices focused talks too. That makes at least seven talks out of 20, which, considering the open process seems pretty healthy. In addition, there is still a real appetite among the delegates for introductory sessions on much of this material; showing new people are becoming involved all the time.

    So why questions about the future?

    While Code Camps seem to be fairly organic about springing up, we have only just begun to see the emergence of regional DDDs in the form of DeveloperDayScotland. So one question that might be asked is whether the perception that the fact that we sell out of spaces within 24 hours, is not indicative of a demand that is not being satisfied by a central event, and it would be better to let many DDDs spring forth across the country, even if each one was smaller and more localized. It might also stop us falling into the ‘usual faces’ trap for speakers, as only a few would probably be dedicated enough to attend all the DDDs, opening the path for more local speakers to emerge.It would seem that the other way is to up the bar for the central event, and fly in top-name speakers. I’m cautious about this model. While you might construct a cracking event, and it might be on a weekend, and free to attend, it would also lose the ‘promote local talent’ and ‘voices from the coalface’ feel that is, for me, at the heart of DDD.There is a part of me that questions if we should not be more like code camps, publish our manifesto for DDD (my suggestion is an agile manifesto style: We would prefer local speakers to ones from further away set of rules).I know that other members of the committee have raised the idea that we need to meet on the future of DDD. So Craig, Zi, Phil, Guy and I will be sitting down to talk about what happens to DDD. However, it is ultimately the community’s conference, so it would be great to hear what you have to say, especially the usually silent majority, before we sit down.   

     

  • All aboard the Monorail

    Last night’s London .NET user group meeting was hosted by Conchango (with Microsoft picking up the pizza). Thanks to Michelle Flynn for making us all so welcome and for her team’s help on the night. Thanks to Daniel Moth for his session too.I promised to share a few links from my Monorail session. Most of the MVC pattern material can be found in the three posts I made on my old blog. You can find links within there to both Monorail and Fowler’s taxonomy of MVC variants.

    In addition I was asked in conversation about implementing an MVP pattern and I mentioned Michael Feather’s article on The Humble Dialog Box and Jean-Paul’s article on MSDN as good introductions to the topic.A straw poll suggested that 90% of the audience could see real benefits to an MVC implementation over the Model-View pattern in WebForms with its leaky abstractions after the implementation, which I think is a pretty good indication of how much cleaner, and simpler, code delivered under this model is.  

  • Which practices to implement first?

     Over on the altnetconf list the question arose as to where you start when implementing agile within an organization. The first thing to bear in mind is that you probably want to avoid implementing all the practices, at the same time. This is likely to cause you to fail, because your attention will be split across too many activities. (Of course this is true for an existing organization where you do not have experienced agile developers. If you are in the fortunate position of putting together a new team with experience agile developers, you are  going to bring your practices on board much more quickly, but you also don't need this advice because you will understand how to get there from experience.)

    The advice from Beck in XP 2nd Ed. on implementing agile is to start with one of the XP practices: "It's easy to start by changing one thing at a time. I think it's hard to jump in and do all the practices, embrace all the values, and apply all the principles in novel circumstances by reading this book and deciding to do it. The technical skills in XP and the attitudes behind them take a while to learn. XP works best when it is done all together, but you need a starting place."

    So where do you start? When trying to bring agile into an organization you want to think about resolving key pain points. Resolving visible pain points using agile practices will demonstrate the value of agile practices and encourage buy-in for further change. What that pain point is will vary by organization.

    Here is how we put agile in place at my current organization. One thing to point out up front though is that in all of these cases, whilst I can identify the starting point, we are continuously improving, so while we may have initiated use of a practice that does not mean we were as good at it as we are today, not does it mean that even today we are perfect on any of those practices. Agile is a process of continual reflection and improvement.

    We started with build, because there was considerable pain around the existing build and configuration management process which was based off building with devenv, using sourcesafe for source control. Build required a full time developer to manage, there were huge versioning and dependency issues and branching off the main code line for a release was a fraught process. A new developer often found it took them two days to set up their box to build locally. We began by pushing NAnt scripts as the preferred build mechanism, re-structuring applications to a standard format to allow us to develop internal build libraries and integrating cruise control so that projects built with any check in. This was not an overnight process and we had to go through a number of iterations, but once your build process is repeatable and automated a lot of the pain goes away. We slowly further improved the build process as developers mastered previous technologies for example swapping SourceSafe for Perforce (Subversion is a good alternative too) to do SCM.

    Next we introduced TDD as soon as we had a new project. hey had a lot of bugs going into QA, and even into live, and some quite byzantine architectures, so we knew we could make things visibly better. It is easier to introduce TDD on a greenfield project because it is easier to get a system under test from scratch. Legacy code projects require the most knowledge to put under test and starting your journey into TDD with them often has low rewards for high effort. That tends to lead to disappointment and frustration among developers that TDD is not helping them, and you will find it hard to encourage them to to continue to practice test-first approaches. Conversely, TDD on a new project will show people the benefits much more clearly and once they are sold on the principle you can look at those legacy projects. I was asked on altnetconf if I would start with TDD if I  had a greenfield project to go when introducing agile, and yes, my experience so far suggests that CI and TDD would be my first port of call in such an environment.

    My experience is that it takes a developer around 6 months to become truly test infected. They will be slower while they learn the techniques, perhaps 25-50% on experience. During that period you will have to pressurize them to write tests first, especially when schedule pressures would otherwise cause them to slip back into hacking their code out. Be prepared for some of the team to feel about you the same way that a recruit feels about his drill sergeant, because you keep banging on about tests. My team rib me now, but I am sure there are periods where they didn't like me. Once you are through the pain, they will swear off developing any other way, even the die hards who said their code did not need tests.

    Then we moved to Scrum and introduced the idea of iterations, stand ups etc. (we had used stand ups earlier without bringing in the rest of Scrum). This one causes conflict with any existing project managers who come from a Prince 2 background, so needs buy in from the top to make them conform. But, by far, the biggest issue here seems to be end user buy-in. Some of our projects still drift into waterscrum with a long deployment phase and UAT period because we cannot get the feedback we want during an iteration. Again its about the users learning the value of their involvement over time.
     
    For deployment we reached a confidence point with our NAnt scripts that we were able to deploy to development and later to test and production environments automatically. As an aside we use XMLPoke to push in values into configuration files that change with each environment, so we just maintain a separate 'include' file for each environment we are targeting, that gets pulled in by the build. We also reached the point of authoring our own NAnt tasks now, so that we can solve local issues, such as uploading document templates for our document generation process, as part of the build.
     
    At this point we tend to use Crystal Clear as the source of our development practices like Exploratory 360, Blitz Planning etc over XP to extend. Personal preference perhaps, although my previous shop was nominally XP and actually didn't look a lot different by the time we had decided what practices were working for us and what weren't. Looking over at the white board our team is currently practicing: TDD, CI, Side-by-side programming, Daily Stand-up, Planning Poker, Burn Down Chart, Coding Standards, Reflection Workshop/ Sprint Review, Exploratory 360, Blitz Planning/Sprint Planning, and Two-week iterations. Of course those practices could change if we don't think they are working for us.
     
    I think the key point is though, you need to look for where agile can surface and solve the pain problems in your current organization. If you can make something better then people will start to view agile as a problem solver and warm to it.

    My experience also suggests that agile is a journey. There is the idealized vision at the end, a perfect XP or Scrum team, but the reality is you are always short here and there, but the struggle toward that idealized model improves your software. We are not there yet. We have lots of shortfalls and keep working to make them better. next off our block is trying to improve our ability to get QA to test within the boundaries of our iteration. So don't be discouraged if you are not perfect, everyone else is just 'on the road' too. If you are on the journey don't get dispondent when it is a hard slog toward a finish line that always seems to be over the next hill. Glance back from time to time, and you will see how far you have come, and feel inspired to pick up your feet and keep marching forward again.


     

  • Hello World

    Thanks to the CodeBetter gang for inviting me to join them on this site. I am flattered, and enormously excited. I thought my first post here ought to be the traditional introduction to who I am. Those of you not interested in the bio might want to see me next time.

    I am a Brit, living and working in London. I have about 16 years of experience delivering Microsoft platform solutions for government, healthcare, and finance. During that time I have worked for the DTI, Reuters, Sungard, Misys and Beazley delivering everything from bespoke enterpise solutions to 'shrink-wrapped' products to thousands of customers. I founded the London .NET User Group and I am an MVP for C#.

    I am firm believer in a statement that Jeremy once made that "There are no smart guys, only us."  I'm just another working stiff who has been lucky enough to have worked with people who steered me in the good directions at the right time. I learn more about becoming a better developer every day from people I work with and from people around me in the community. I want to share those experiences with you. My hope is that some of you will be inspired by that to join me on the journey to becoming a better developer. I'm sure that many of you who do will quickly eclipse me, and I'll welcome that, because then I will learn from you.

    Feel free to take a look at my old blog if you want to see some of the things I am likely to talk about. Google seems to bring people to my old site for this post on LINQ to SQL and Persistence Ignorance, and perversly problems people seem to have with their installers. I'm prouder of the former and it represents the future direction of my posting more, but it just goes to show.

    Oh, and Scott can't understand my accent, so he has no idea what I'm saying to him.

     

More Posts

Our Sponsors

Free Tech Publications