The role of Team System in a Software Factory

29 02 2008

I’m no thought leader on the future of IT, but I do have a vision of how software development will evolve in the next couple (or hundreds) of years and Team system plays a big part in it. But before we dive into this vision, let’s have a look at what the development world looks like today.

What does software development look like today?

Today, software development is a craftsmanship, a lot like conventional industry was 400 years ago. Most software development teams are made up of a small amount of specialist developers, of which only a few actually master most of the available technologies or as Bart calls them in his article ‘The developer gap’, expert and regular developers.

In some cases these teams are self organizing, which means that they have managed to gather the right mix of developers, each talented in their own way, so that they can make all the necessary decisions for themselves. If you are in such a team, great, but reality tells me…

That there are just not enough good developers on the market to be sure that you can compose a self organizing team in the limited time you have to set up the project. Often you just have to work with less talented people, or people whose talents don’t match with the needs of the project.

And if you do manage to find the expert developers, you will only find a few and they will be pretty expensive. Both of these factors make that the software, which is supposed to add value to the business, will take several months to develop and will cost a lot of money. By the time the software is released, the market will already look differently and a lot of the effort goes to waste.

Don’t get me wrong, I like the idea of the self organizing team, and I would love to have a few thousands of colleagues that have as much expertise as the few dozen I currently have, but that’s just not how our market works. And trust me, after being in the Compuware recruiting team for a few years now, there are not many good developers available on the market.

Where should the industry go to?

In my opinion, the total cost of software development and the lack of talent on the market will force us to hire low cost, less skilled workers to complete our projects. We still have to go through the same revolution as happened with conventional industry in the 1800’s, we need to improve the current processes of software development and automate them into factories.

The biggest difference between IT and conventional industry, however, is that we are not developing the same parts all over again, which makes standardization a lot harder. Each project has different business needs and is started to solve a different problem, it will be very difficult to design a ‘Customer’ class that will fulfill the needs of a local grocery shop or that of your countries biggest transport company.

I don’t believe we can make some ‘Robot’ that will generate all the code of an application for us; instead I believe we should organize our software factory around a development process for which some tasks are fully automated, a few are done by expert developers and most of the labor is done by less skilled people, who don’t necessarily have to have skills in software development.

What needs to happen before we can achieve this?

As the state of software development is today, I don’t believe this revolution is at hand in a very short time. A lot needs to be changed before we can even think about large scale automation:

First of all, we need to find the default software architecture. If we want to automate the development of most types of software, we need to figure out what the default architecture is for each kind of application and define the common denominator. Today most teams start all over again for each software project they attempt, but to be productive they should start off with the default architecture and only make minor changes to it if a special feature is required. I believe this is probably the hardest task we are facing; I do have a default architecture for services, one for smart clients and one for web applications, but to achieve this goal all kinds of applications should be architected the same way preferably. Multiple defaults are possible as well, but that would make reuse of the assets much harder.

Secondly, all software components should be standardized. This means they all should conform to a certain design, a fixed interface. In transport, all containers fulfill to a number of standard sizes, no matter what can be transported with the container, no matter how it’s going to be transported. This way the containers are interchangeable allowing trucks, trains and boats to be designed without knowing what they are going to be used for. Once the interfaces of all software components are stabilized, general infrastructure code can be provided to allow faster and easier development.

Finally, each detail of the software development process, or better, the entire application lifecycle, should be studied and documented in detail. When a car manufacturer starts his assembly line, he will know exactly what parts are used at which points of the line, he knows how many milliseconds it will take to put a screw into the chassis, each part of the assembly line is controlled and monitored with great detail. When all details are documented, they can be assigned on an individual basis to non developers as well, in my opinion. With the correct documentation I do believe that a non developer can perform a code review targeting coding guidelines for example.

What technologies are required?

In order to automate some of the tasks, we can resort to code generation. However I’m not a big believer of code generation as each piece of code to be generated has its own specific needs. What I do believe in though is discrete code generation in the form of snippets and GAT recipes that can be used by less skilled developers to perform a specific task like creating a data access component for example.

If we would like to use non developers to actually create parts of the application, we need to provide them with a DSL so that they can configure the component with terms they know and have the code generated when they are finished.

But a software factory is more than just snippets, recipes and DSL’s. Each of these technologies is like a robot in a factory that can perform a small specific task. All of these robots should be combined with people (both expert developers as unskilled ones), to form an assembly line. The role of the assembly line can in my opinion be played by Team Foundation Server, and more specifically the process template and continuous integration features.

Team Foundation Server

Besides all the necessary tools that TFS provides us, like work item tracking, version control and a build server, it has one big advantage over different products: The process template. The process template gives us the means to set up the configuration of an assembly line.

We can create custom work items that can be interpreted by humans to perform a task, but using the work item object model, software components can interpret them as well and could automate some of the jobs. Obviously a standardized and well controlled process is crucial to achieve this.

Work items can be linked to code, in order to keep track of the time it took to complete the work, so that we can monitor how the assembly line is doing. Check-in policies allow us to enforce the linkage so that human interactions can be tracked as well.

Team build and more specifically MS Build allows us to perform automated compilation, testing, code metrics collection and automated deployment without knowledge of the developer checking in his code. With the TFS 2008 features and the SDC library, this task becomes even trivial.

All collected information is gathered in a data ware house so that the management team has a clear view of performance and quality at any moment through advanced reports.

My plans in the near future

I hope I have convinced you that TFS, or one of its successors, will be the key to the future software factory that I have in mind. In the near future I hope to write a series of articles on several parts of TFS and how you can use it to automate some of the tasks that need to be performed during the development of an application.

I will double post the articles that are directly related to software factories on both my own blog and the intofactories blog. Articles that cover only technical details of TFS like the build API or process customization will only be posted on my own blog so that IntoFactories stays focused on software factories only.

Stay tuned,
Yves Goeleven


Actions

Information

8 responses to “The role of Team System in a Software Factory”

1 03 2008
Nico Mommaerts (01:02:25) :

Hey Yves,

while the prospect of having a software factory as you explain it certainly seems promising I have some reservations about some of the things you say. My biggest gripe about your vision of the software development industry is that you think we should let less skilled developers do all the standardized labor and let more experienced developers do ‘the thinking’. I think this is a very Taylorism-like approach to software development. I’d rather hope we will skip the revolution of the 1800’s and go straight to the one of the +1950’s in which strong leadership will elevate ordinary developers to achieve stellar results. (Poppendieck on Leadership, PDF Link)

As P.Brooks says in his book The Mythical Man Month, one can divide the difficulties of software development into two kinds: essence, the difficulties inherent in the nature of software, and accidents, those difficulties that today attend its production but that are not inherent. The complexity of software is an essential property, not an accidental one.

While I do believe and agree with you that software factories can alleviate some difficulties from the software development process, I’m convinced that the difficulties they solve are only accidental ones. Imho you can only tackle the essential difficulties by building a great team. I do agree you need some star developers or designers, but I think you are putting too much emphasis on ‘dumbing down’ work to tasks less skilled developers can tackle, an idea which you seem to enforce through your assembly line analogy.

Having said that, I enjoyed reading your post and I’ll certainly stay tuned,
Cheers, Nico

1 03 2008
Goeleven Yves (18:01:57) :

Nico,

I don’t think we ’should’ hire unskilled workers, I believe we will be forced because of the lack of talent on the market and the ever increasing demand for faster software development.

To be honest, don’t you think most teams today allready have one or more unskilled person in their midst, like a Cobol developer for example that hasn’t really mastered that new paradigm called OO?

I think our future processes must accomodate to this evolution so that development can continue even when we have to resort to unskilled workers just to get the work done in a timely fashion…

Furthermore, I didn’t intend to give the impression that I wanted to ‘dumb’ the work down, instead i believe we should make a better analysis of our workflow, so that we know about all the details involved in creating a certain piece of the solution (be it a data access class, a piece of documentation or anything else…). This way we can extract those parts of the workflow that can actually be done by lesser skilled people (for example, managing some configuration files and packaging software into installer packages or a person who is dedicated to enforce the branching and merging strategy).

1 03 2008
Jan Van Ryswyck (20:03:56) :

Hi Yves,

I’m with Nico on this one (surprise, surprise). I already wrote down my take, so I’m not going to repeat myself. I just want to add, that I believe in the people I work with, whether they are talented or not. Call me naive, but I’ve seen this work. I’ve been involved in the recruitment process in the past. The one thing I’m looking for in a developer is passion, not whether he knows some insignificant piece of the .NET Framework that is going to be obsolete with the next release anyway.

I don’t like to put people in boxes. Software developers want to be creative and involved if you just let them. When you take this away, you are practically killing the team. Being an architect or a team lead, is all about finding the right “path” for the team.

There is no such thing as a default or reference architecture. Being dogmatic about anything is not going to work. There are nog silver bullets, there is no universal thruth. Something that works for application X doesn’t necessarily works for application Y.

To wrap this up, I would like to quote the first sentence of the agile manifesto:

Individuals and interactions over processes and tools

That’s what I like about being “agile”.

PS: Unfortunately, I’m not part of a self-organizing team (yet). It’s still one of my goals to achieve this for my current team and I’m 200% convinced that we as a team can pull this off.

1 03 2008
Goeleven Yves (20:51:20) :

If anyone is in a self organizing team, please let me know. This is exactly the problem. A self organizing team is an ideal situation, not something I’ve seen in reality up until now, and I’m in the business for quite a while now.

I agree that we are all looking for people who have the potential, this means they can become the star developer of the team. As one of the compuware recruiters, I can tell you that these kind of people is all we are looking for, yet hard to find…

All that I’m saying is that this situation, this ideal situation, can’t be maintained for a long period of time with the current availabillity of people on the market… and that we should be prepared for it…

2 03 2008
Jan Van Ryswyck (11:23:31) :

Although I recognize that self organizing teams are not the majority, the current situation is not working either. This situation where there is one lead/architect who is setting up the rules of the game without any form of communication/coaching for those poor soules that need to use them, has a name: it’s called ‘drive-by architecture’. Again, this doesn’t work either.

I’m convinced that we should bite the bullet and choose the hard way (introducing self-organizing teams) instead of choosing an easier path. A mind shift is needed instead of throwing more rules and ‘tools for fools’ at them.

In every branche, there are people who are good at what they do and there are people who are not so good at what they do. You need to accept this. I see this mostly as a HR problem. Taking someone from the streets, making him follow a course for two weeks an putting him behind some designer-like-development- environment where he can drag & drop himself to glory is a doom scenario. This is only going to create lots and lots of unmaintainable VB6-like applications, which is surely going to cost lots and lots of money. This is certainly not the direction where this industry should be going either.

2 03 2008
Goeleven Yves (11:44:03) :

Taking someone from the streets, making him follow a course for two weeks an putting him behind some designer-like-development- environment where he can drag & drop himself to glory is a doom scenario.

I totally agree with you that with the current understanding of development processes, doom would be upon us, but my opinion is that in a number of years / decades we will have to figure out how to enable this possibility, even if we don’t like this… and trust me, I’m not jumping for joy either.

But let’s agree to disagree… What I do believe we agree on is that integrating GAT recipes with Team System seems really appealing. Wouldn’t it be nice to just create a work item to, for example, create a new solution. When the workitem is created, the workflow of TFS enables some automated process to actually do the work for us via a GAT recipe? It would definitly ease my working day…

2 03 2008
Jan Van Ryswyck (12:39:07) :

Indeed, I agree that we disagree in a respectfull way ;-) . As I already lined out in my blog post, I like what Bart already mentioned about code snippets, project templates, etc. …. I do believe this can make you more productive because they are on a micro-level. I do not believe in full blown software factories, but I’m not shielding myself from this idea. That’s why I’m reading this blog and love to argue about this.

Anyhow, have a nice weekend and a productive workweek :-) .

3 03 2008
Steve Degosserie (09:19:10) :

Individuals and interactions over processes and tools … yes, when your development team(s) are stable.

If you have a high turnover, and must replace experienced people leaving your company, by unexperienced developers, then Software Factories do make sense, as well as a ‘default architecture’ or, at least, default components re-usable in different architectures, along with procedures & ‘recipes’.

Also, one cannot expect to find passion in every software developers … some have passion, some, who work to pay their bills onyl, don’t.

In the end, you have to ask yourself ‘What is more harmfull … losing an important experienced developer, or losing your company know-how ?’ (also know as the ‘Truck Factor’ … what if your super genius architect is killed by a truck ? can your current project survive and reach the release ?)

So, I think it is a safe bet to industrialize important parts of you software development process, with key architects / developers coaching, teaching & ensuring that the processes are correctly followed by less knowledgable ones.

Leave a comment

You can use these tags : <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>



Warning: include(news_rss.php): failed to open stream: No such file or directory in /home/mettheat/public_html/intofactories.net/2008/02/29/the-role-of-team-system-in-a-software-factory/index.php on line 604

Warning: include(news_rss.php): failed to open stream: No such file or directory in /home/mettheat/public_html/intofactories.net/2008/02/29/the-role-of-team-system-in-a-software-factory/index.php on line 604

Warning: include(): Failed opening 'news_rss.php' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/mettheat/public_html/intofactories.net/2008/02/29/the-role-of-team-system-in-a-software-factory/index.php on line 604

All News