Here is a little trick that allows having connections to multiple databases with pure ActiveRecord.

First, a little recap. Here is the code for making a single connection:

require 'active_record'

ActiveRecord::Base.establish_connection(
  :adapter  => 'mysql',
  :database => $env[:db_name],
  :username => $env[:db_user],
  :password => $env[:db_pass],
  :host     => $env[:db_host])

class MyTable < ActiveRecord::Base
end

The nice thing with this approach is that any additional class will automatically reuse the defined connection.

However, if we need to have multiple connections the approach is a little different:

require 'active_record'

ActiveRecord::Base.configurations["db1"] = {
  :adapter  => 'mysql',
  :database => $env[:db1_name],
  :username => $env[:db1_user],
  :password => $env[:db1_pass],
  :host     => $env[:db1_host]
}

ActiveRecord::Base.configurations["db2"] = {
  :adapter  => 'mysql',
  :database => $env[:db2_name],
  :username => $env[:db2_user],
  :password => $env[:db2_pass],
  :host     => $env[:db2_host]
}

class Table1 < ActiveRecord::Base
  establish_connection "db1"
end

class Table2 < ActiveRecord::Base
  establish_connection "db2"
end

This approach allows defining any number of database connections and any number of tables on those connection. It also allows adding connections and classes dynamically based on some runtime information. The only drawback is that in each class you are required to define which connection needs to be used.

I suppose exaggeration has a long tradition in résumé writing and interviewing, but I still find myself surprised by candidates who apparently think they can bluff their way through an interview just by parrotting the buzzwords they read in the job announcement. Sometimes it's kind of obvious when a candidate has a good grasp of the job requirements. Sometimes...not so much.

I favor letting candidates for programming or testing jobs show what they can do. Typically, they prefer to work than to talk anyway. Well, the good ones do. Besides, it's gotten to the point that we can't tell anything at all from a candidate's résumé, and if a candidate shines in an interview it may only be because he/she has had a lot of practice interviewing.

Candidates need to find out whether the team they would join is a good fit for them, just as the team needs to get to know new people a bit before making a commitment. It's only fair for everyone. So, if we're going to audition technical candidates and we need to ensure they are a good fit for the team, it follows that the audition should consist of hands-on work with some of the team members. The closer the audition can be to the real work context, the better.

Companies like (for instance) Lean Dog Software and Emergn audition candidates for hands-on technical jobs, and the result is evident in the quality of their services. When I can, I advise clients to use auditioning in addition to interviewing as a way to screen candidates. It usually works quite well, although sometimes they have to reject a few who might have slipped through an interview-only recruitment process.

Early in my current project we were looking for a couple of decent Java programmers to round out the team. We asked for candidates who had the usual Java skills, plus a bit of Oracle, and if possible some exposure to development methods such as test-driven development and pair programming. We weren't asking for gurus; just for competent people who were interested in a collaborative working style.

Several candidates made it through the initial phone screen and came to the office to meet the team and have an interview. They claimed to have between 9 years and 18 years experience with Java. Think for a moment about those numbers. While there is a certain amount of complexity in the practice of software development, it is definitely not rocket science. Nine years experience is plenty of time for a motivated individual to reach a solid Practitioner level, and in some cases a Journeyman level. By the same token, anyone with 18 years experience who has not achieved the Practitioner level is in the wrong line of work. So, I don't think our expectations were unreasonable.

All these candidates claimed strong Oracle experience. All claimed to be Java experts. All claimed to have significant experience in TDD and pairing, spanning multiple projects and teams over a period of several years. During their interviews, all the candidates were able to describe both technical and process issues accurately, if dryly. Let me reiterate that we did not insist on previous experience with TDD and pairing; we understood that these skills are the exception rather than the rule, circa 2010. The desire to work in a collaborative style and a willingness to try unfamiliar techniques would have been perfectly acceptable traits for the new team members. That assumes they had basic competence in SQL and Java, of course. Even in those areas our expectations were not extreme.

I was a bit suspicious when the candidates were unable to tell any personal war stories from their experience; they merely recited textbook definitions of things like "inheritance" and "unit testing." Although each of them listed TDD and pairing on their résumés, none was able to describe any cases when they personally applied TDD or pairing. They weren't able to share any funny or memorable tales from the trenches. Can you really work for 18 years and have no personal war stories? Nothing interesting ever happened? Nothing amusing? Nothing ironic? Nothing surprising? Nothing edifying? Nothing discouraging? Nothing stupid? Nothing ever happened? Nothing at all? <gesture type="chin-scratching">Hmm. Spidey sense tingling</gesture>.

On to the audition phase. Each candidate paired with two team members, one at a time, while the other team member and I observed. We explained that we wanted them to act just as if they were already on the team and they were working on a User Story with a partner. We wanted to see how they interacted with their partners and their teammates, and how they approached test-driving some code. Some of the following description uses the plural; in fact each candidate was interviewed and auditioned individually. It's just that they didn't behave any differently, so the plural is a valid way to describe what they did.

I want to be perfectly clear about this: We never asked a candidate to demonstrate any skill he had not claimed to possess. Had a candidate said that he had never tried TDD before, then the tenor of the audition phase would have been different. For instance, we might have shown him the TDD cycle in the process of getting a sense of his general Java skills. But these candidates claimed that they had been using TDD for years already, and that they were very good at it. If that were true, then it should have been both easy and enjoyable for them to show us.

The project involves Java and Oracle work. We started each candidate with a simple SQL exercise. Given Employee and Department tables, they were asked to write a query to list all employees by department, including only those employees who were assigned to a department. Then, they were to write a query showing all employees by department including those who were not assigned to a department. So, first an inner join and then an outer join. That's all. They were presented with a sqlplus command line where describe commands had already been entered for the two tables, so they could see the table structure from the start.

No candidate interacted with his pairing partner at all. None uttered a single word unless his partner prodded him with leading questions. No candidate bothered to run a 'select *' to see what data was in the two tables, so they would have some idea of what correct output might look like. No candidate remembered the correct syntax off the top of his head; not a problem in itself, except that no candidate bothered to ask his partner, look for a manual, or search for the answer online.

After seeing the first candidate do nothing at all for a solid three minutes, we decided to open a browser to Google and leave it there, on the screen alongside the sqlplus window, as part of the set-up for the audition. (Psst: Hint, hint.) All the candidates just sat there. At one point I said, "You know, a job is not a university exam. On the job you can Google for the syntax." That guy Googled and completed the exercise. We were thrilled that he was able to use Google, but less than thrilled at the fact he still never interacted with his pairing partner.

After a few minutes we moved on to the Java exercise. For this, we set up a simple problem to build a 'stack' class. Candidates were presented with Eclipse, already open to the project, and with a unit test class already open in the editor. The unit test class looked like this:

public class TestStackOriginal{


    @Test
    public void whenStackIsEmpty_popReturnsNull() {
        OurStack stack = new OurStack(); // Note: This is NOT java.util.Stack
        assertNull("Should have returned null!", stack.pop());
    }

    @Test
    public void whenStackHasOneItem_popReturnsThatItem() {
    }

    @Test
    public void whenStackHasTwoElements_popReturnsTheLastElementAdded() {
        throw new RuntimeException("Write me!");
    }

    @Test
    public void whenStackIsEmpty_pushAddsOneItemToTheStack() {
        throw new RuntimeException("Write me!");
    }

    @Test
    public void whenStackHasOneItem_pushAddsASecondItem() {
        throw new RuntimeException("Write me!");
    }

    @Test
    public void whenStackHasTwoItems_peekReturnsTheTopItemWithoutPoppingIt() {
        throw new RuntimeException("Write me!");
    }

}

The problem is simple enough that candidates ought not become confused by the "requirements." The test cases pretty clearly suggest what to do first. The exercise presents several opportunities for candidates to show their knowledge and demonstrate how well they interact with pair partners. Personally, I had looked forward to this. I expected advanced practitioners would have something to say about the pop method returning a null reference and, possibly, about the fact that one of the test cases will pass even with nothing added to it.

Yes, it would have been nice to hear some discussion of these things, and maybe more. But what we were most interested in was the way in which the candidates worked with their partner, and that they seemed to understand the TDD cycle. We didn't pull these things out of our...well, out of thin air. All candidates claimed to have significant experience with TDD and pairing, as well as with Java. Therefore, it was reasonable to expect them to know how to function in a pair, how to drive code from unit tests, and how to write Java. Since they presented themselves as "senior" Java developers, we expected them to have some knowledge and opinions about OO design, too.

No candidate interacted with his pairing partner at all. The first wrote unit tests for java.util.Stack, without asking anyone if that was the point. (That's the reason we added the comment in the test case and suggested the class name OurStack.) He claimed to have been doing TDD for the past eight years. One would expect that TDD was his default mode of work. He gave no indication that this was true, or even that he had the slightest awareness that the exercise had something to do with TDD.

Another candidate immediately changed the implied method signature of the new stack class so that it threw an exception instead of returning a null reference. He did not consult his pairing partner about it. It did not occur to him that the team may have decided to return a null reference before he joined the project. He just went ahead and changed it — in a utility class, no less — without asking about it. Yeah, I want that guy on my team. Always a surprise in store. Who doesn't like surprises?

No candidate questioned the null reference. While throwing an exception might be overkill for an empty stack condition (even though that's what java.util.Stack does), returning a null reference isn't a good practice, either. The exercise was set up that way to provide an opening for candidates to show that they were aware of this, and to suggest an alternative such as the Null Object Pattern. No candidate provided any evidence that he was aware of any of this, despite claimed experience in the range of 9 to 18 years and self-assigned titles of Senior This and Senior That.

One of the few candidates who actually tried to write some code instead of just staring off into space implemented the pop method as follows (after his partner suggested that he actually write something instead of just staring off into space):

public String pop() {
    ArrayList stack = new ArrayList();
    if (stack.size() == 0) {
        return null;
    }
    return null;
}

He was then unable to explain how the code worked. His partner asked him about the effect of the size check. Blank stare. His partner asked him what would happen if the stack contained an entry before pop was called. Blank stare. Nine years of Java experience, by the way. Engineer. Senior Something. Sharp-looking suit. Classy necktie. Yeah.

That was the best result achieved by any of the candidates.

You can't fake this stuff. Don't try. It's a waste of everyone's time, and all you will gain is a reputation as a liar.

Just back from Lean & Kanban Europe 2010. The first order of business has to be to thank Maarten Volders, the main organizer of the conference, for a job well done. It was only the second event he has organized, and the result was as good as conferences organized by more-experienced people. The quality of the conference food was exceptional, compared with any other conference I've attended; and food is important! For next time, a commemorative T-shirt would be nice.

Alan Shalloway's insightful opening keynote set the stage for a great conference full of practical learning opportunities and rich interaction among the participants. Good summary of how and why current leading-edge thinking about process improvement came to be.

I've been wanting to get more directly involved with the Lean and Kanban community for some time now, and this was the first explicitly Lean-focused conference I've been able to attend. There is considerable overlap between "agile" and "lean" in software development. Many people who are already well known in the agile community have become active with Lean and Kanban in the past couple of years. It seems to be a logical next step in the evolution of software delivery practice.

A mindset of continuous improvement is fundamental to Lean thinking. That mindset was very evident in the nature of the presentations given by the Big Fish in the Small Pond of Lean and Kanban. They were not so much "experts" laying down the law for "followers" as they were peers offering experience reports bolstered by thoughtful analysis. There was a lot of straight talk from the trenches, deep interest in understanding what worked or didn't work and exactly why, and a sincere desire to learn rather than to profess.

A great example of this was David Anderson's talk, "Using Classes of Service with Kanban Systems for Improved Customer Satisfaction." Although the talk was not listed on the "experience report" track as such, it was entirely based on practical experience. He described a situation from a real-world project that led to the idea of classes of service, which I suppose you could say are different types of work requests. A key point is that the types or classes are not arbitrary. A basic concept of Lean development is that delay is a cause of many problems. Classes of service can be defined on the basis of the cost of delay curve, a method of understanding the business impact of delayed delivery.

The cost of delay curve for various types of work requests leads directly to four different classes of service: Expedite, Fixed Delivery Date, Standard Class, and Intangible Class. I will refer you to published descriptions of these rather than reiterating the material here. David's teams have set different work-in-process (WIP) limits to each class of service, with Expedite limited to 1. That limit prevents everything from being defined as Expedite, which would throw the whole process out of whack. He also mentioned that peer pressure keeps the number of Expedite requests low, since client managers have to justify their requests to the other stakeholders.

The classes of service concept is described in this blog post by Jeff Anderson, in this set of slides from David's presentation at Agile 2009 (pdf), and in David's book, Kanban.

Mary Poppendieck gave another sort of experience report as a way to explain the concept of pull in her talk, "What is this thing called 'pull'?," She told the story of a time when she was a line manager at a plant that manufactured cassette tape cartridges. The day came when Japanese competitors were able to sell cassettes at half the price as her own company's cost to manufacture them. In those days, there were no books or conferences about the Toyota Processing System, and the word "lean" had not yet been coined.

She described, step by step, how the company came around to realizing that the only people who could devise a production system that could compete with the Japanese were the people who did the actual work on the shop floor. No system could be designed by management or by outsiders that would work. What they ultimately came up with was a pull-based kanban system similar to those used at Toyota. Something to take away from this is the fact that all this "lean" stuff is not ivory tower academic theory; it comes directly from practical experience in competitive business situations where failure equates to direct financial losses.

John Seddon's closing keynote on Thursday was refreshing, entertaining, and informative. His delivery is low-key, engaging, humorous, and British. He spoke without slides, and told the story of a consulting engagement in which he helped a service company improve its performance. It seems the company was in the home repair business. They took calls from residents who reported needed repairs, determined the urgency of the repair, and dispatched workers to carry out the repairs. Customer service was dismal.

The short version is that he advised the company first to improve its process, and only then to engage IT people to automate it. In his experience (and probably in most of ours, too), when IT is brought in at the start of a process improvement initiative, they tend to (a) automate the current process, and (b) make the solution too complicated. The result is the same old process, without any improvement, a huge bill for software development services, and an automated solution that doesn't serve. By improving their process first, the company had a greatly simplified and highly appropriate process in place before they asked for a bid for automation. A programmer offered to build the solution in three weeks for £2,000. The company offered to pay £3,000 for a solution delivered in two weeks. They got one. It worked. They're still using it.

I wasn't sure how well-received my own session might be. It represented some out-of-the-box thinking that very few of the people I've worked with in the past five years would consider feasible or even sane. It was difficult to assemble the ideas into a coherent enough form for a presentation, and I'd had the opportunity to dry-run it only with one colleague prior to the conference. She agrees with many of the key ideas, so I didn't get any feedback of the sort that could protect me from self-annihilation, such as "Are you crazy? You can't say that! That's nonsense!" I had no choice but to expose myself to that sort of feedback with the camera rolling and in front of a room full of people who know more about Lean than I do. Oh, well. Worst case: I might learn something.

I was pleasantly surprised by the positive response. It seems others also have been questioning the conventional separation of IT and "business." I was also pleased to note that about half the participants had read Implementing Beyond Budgeting and understood the impact of the annual budgeting cycle on an organization's ability to embrace Lean thinking. Several people in the room and at the conference have already been developing ways to apply Real Options to software delivery problems.

A question that came up, and that often comes up in conversations about the idea of integrating IT with business units, was how to manage economies of scale, governance, and shared IT assets if we fold IT functions into business units. My view is that the only portion of IT services that should be folded into the business is application development and support; that probably accounts for around 20% of the work of most corporate IT departments. Central IT functions should remain centralized. So, we're not talking about ripping the IT department to shreds.

Frankly, this seems rather obvious to me, and yet many people I talk to about the idea of integration seem to assume it's all or nothing, and folding IT into the business means the end of the central IT department. That isn't quite what I mean by integration, and I'm glad to have had an opportunity to clarify that with a sizeable number of people.

I suppose I was surprised by the positive response because in my recent experience I haven't met many people who are aware of these issues or who seem to perceive organizational structure as an inhibiting factor in their process improvement efforts. Very few are genuinely interested in change at all, and those tend to think about improvement strictly within the scope of the IT department or individual software development projects and teams.

It was very refreshing to hear positive comments about the session afterwards. I had almost given up on the idea that organizations can be changed at all. I don't know whether it's because the conference concentrated people together who share a common interest in general process improvement, or because North American companies are too stodgy and traditional-minded to consider change of that scope. I suspect it may be a little of both. Either way, I was very encouraged and re-energized by the conference. Really glad I participated!

There's someone else I think should be acknowledged. He wasn't on the program, and I don't think he often presents at conferences: Paul Dolman-Darrall, who works for Emergn. He participated in my session about the future of agile in Orlando, and again in my session in Antwerp as well as a couple of the other sessions I attended. He always brings fresh thinking to the table, and he has a knack for perceiving connections between ideas that other people overlook. His questions and comments always help refine, redirect, or simply correct the assertions made by speakers. He's an avid reader and always suggests books and articles for us to follow up on. He's also hands-on in the field doing lean and agile coaching and delivery, so his thoughts are not purely theoretical. This is just the sort of person I like to have in my sessions. It helps make the sessions into powerful vehicles for learning.

Unfortunately I had to miss the final day of the conference. Several compelling sessions took place that day, featuring a number of speakers who are thought leaders in the Lean software community.

Some pics...

David Anderson's session on classes of service Attendees enjoy the fine weather between sessions Old cranes at the conference venue (used to be a dock) View along the Scheldt from the venue View along the Scheldt from the venue View along the Scheldt from the venue Shopping center near my hotel Street near my hotel Street near my hotel Antwerp Central Station - interior Antwerp Central Station - exterior Paced by another aircraft, approaching Maritimes on the way home

Attending the Lean & Kanban 2010 conference this week, where I'll be hosting a discussion based on the following premise: The existence of an IT portfolio separate from the overarching enterprise portfolio limits the organization's ability to implement and benefit from a lean model. It's on the lean side of the hall, and not the kanban side.

To set some context: The session deals with internal IT departments in companies that do something for a living other than software development or IT services. It deals with the sort of company I think of as a tertiary technology company; that is, a user of information technology, not a creator of information technology. So, if you're tempted to ask, "How does this apply to us? We're a seven-person start-up building a cool new peer media sharing app," don't ask. This isn't about you.

The basic idea is that structure begets function. If your organization is structured in a way that makes lean adoption difficult, then lean adoption will be...well, difficult. The "hook" about the IT portfolio is a conversation-starter meant to underscore the idea that having two separate plans for two separate organizations within an enterprise leads to various manifestations of the Three M's of lean.

I'm hoping that the discussion will progress beyond just the IT portfolio, and that we will be able to explore the ways in which keeping the IT organization separate from the rest of the enterprise inhibits emerging ideas like Real Options and Beyond Budgeting, as well as other ideas that are compatible with lean thinking.

I also want to suggest that treating the internal IT department as a cost center with a fixed annual budget leads to wasteful behaviors, including small-scale time-based estimation, tracking actual time against estimated time per task, gaming the budget numbers, and defining performance incentives for IT personnel that are different from those of the rest of the enterprise, leading to different and independent priorities.

Many organizations use a quadrant-style model for mapping business objectives along two axes, such as "importance" vs "urgency," or "market differentiating" vs. "mission critical." I make the case that when enterprise planning and IT planning are separate, the axes along which planners map their objectives may be different. For example, IT planners might use a quadrant-style planning model with the axes, "value to the business" vs. "risk." Those axes sound good, but all too often "value to the business" merely represents the IT department's best guess, and "risk" merely represents the anticipated difficulty of implementation. IMHO avoiding work that it hard to do, but that supports the enterprise strategic plan, is just a blame-avoidance strategy.

Although planners speak of "alignment," as long as they are working from two different plans the IT organization will never be aligned with the enterprise strategy. When the IT department is treated as a cost center with an annual budget allocation, the tendency is for IT planning to be aimed at minimizing the blame that may be assigned to the IT department for any "failed" initiatives. Perfecting this sort of planning, as recommended by many proponents of formal IT Porfolio Management (ITPM) methods, amounts to "doing the wrong thing better."

The IT portfolio approach also impedes the organization's ability to establish a pull system at the level of enterprise capabilities, because it tends to lead to a program of discrete projects. Each project is, in effect, a batch of work. Some batches are many months long and comprise a very large amount of work. The IT portfolio itself establishes a push system for the discrete projects. High "risk" (that is, "difficult") projects may be declined in order to avoid blame, regardless of their importance to the enterprise strategic plan. The three M's are to be found in abundance in this sort of arrangement.

A common sort of disconnection between enterprise planning and IT planning has to do with needs that are obvious to the IT staff but that do not fall out from enterprise strategic planning. An example is a "technology refresh" initiative, by which I mean a project to replace aging technical infrastructure, a hardware platform, or a COTS package or legacy application. The business risk of leaving the old technology in place grows year by year, but often it is not obvious to business planners. Typically, IT staff try to sell the idea up the management hierarchy, with mixed results.

The same disconnection can occur from the other direction. When enterprise strategic planners identify a business capability that requires an investment in IT infrastructure above and beyond the usual annual budget allocation, they may simply assume the IT department has the capacity to complete the work. Unless the CIO is actually treated as a peer of the CEO, CFO, and COO, there will be no voice in the enterprise strategic planning process that can identify business capability goals that have significant implications for IT.

Another concept I'd like to explore in a lean light is "shadow IT." Given that business people have absolutely no interest in spending money to create their own mini-IT departments, what prompts them to do so? My observation is that they do it because they are driven to do it by IT departments that provide poor service. I think this is another effect of organizational structure and of separate IT planning, budgeting, and tracking.

Shadow IT isn't the answer to the problem, because it amounts to duplication, a form of waste. It can also lead to legal and financial problems for the enterprise, since shadow IT solutions may not comply with regulatory requirements or information security standards. But what if business units normally hired and managed their own application development teams, while the central IT department took care of matters that rightfully belong under central control? There's a lot to say about this and not much room here, but we'll be discussing it this week.

If you're in the neighborhood, drop in and participate. There are a lot of good sessions on the program and several well-known speakers who are thought-leaders in the application of lean thinking to software development and IT management.

There seems to be a growing backlash among software developers against what they perceive to be "agile." It's understandable — in the past decade or so, that notorious buzzword has traveled the world, leaving in its wake results ranging from the astonishingly good to the maddeningly bad. Few people have bothered to analyze the reasons for the results they have seen, or even to try and understand how well they applied agile principles. It's easier to blame a buzzword or a consultant than it is to ask questions that may have answers we don't want to hear.

There's a pattern I've seen repeated over the years. When a bright new idea appears on the scene, people jump on the bandwagon enthusiastically with high hopes. Then, when the shine wears off, the same people are eager to discard the idea and look for something else. I'm beginning to think a lot of people are actually looking for something that will magically solve all their problems without demanding any effort or understanding on their part. They (seem to) want to be told what to do, step by step, by a process that will guarantee success every time, automatically; a process that can't be "done wrong," no matter what; a process that takes the blame for bad outcomes but gives the people the credit for good outcomes. "Agile" software development is at that stage today. The shine is off.

What worries me about this is not that I agree or disagree with individual professional choices about how to build software. What worries me is the final part of the repeating pattern of enthusiastic adoption followed by sneering rejection — at the point when people are ready to discard an idea, they discard everything they perceive to be associated with the idea. In effect, they throw out the baby with the bathwater.

A number of sound software engineering practices have become popularized through the agile movement. The practices aren't part of the definition of "agile;" the Agile Manifesto doesn't mention any specific software development practices. Nevertheless, many people associate certain practices with "agile" because they first learned about the practices in the context of an agile adoption or agile pilot project. Possibly for the same reason, proponents of certain practices tend to label them as "agile practices," too. That probably doesn't help people parse the useful ideas from the general background noise.

The agile movement raised awareness of several good practices, including continuous integration, automated testing, test-driven development, team collocation, and pair programming. You can probably think of more examples. In their eagerness to reject "agile," a growing chorus of voices has begun to chant the mantra that these practices are not valuable.

Throughout the first 25 years of my career, I wrote software in the same way as detractors of TDD write it today: I would quickly type in some code, and then spend the next couple of days poking at it manually, tracking it through a debugger, displaying variables on the console, and anything else I could think of to make it work (sort of). Testing (if any) was manual testing done after the fact, and the smallest scope of a test case was usually an end-to-end functional test.

So, 5 to 10 percent of my time was spent in creative activities like analysis, design, and coding. The rest was spent in debugging and in responding to bug reports. It was a frustrating, uncreative, and boring way to spend my time. That's why I was looking at franchise brochures and considering dropping out of the IT field altogether. Selling venetian blinds or running a storefront for a package shipping company would have been a more satisfying professional life.

When I learned TDD in 2002, it revitalized my enthusiasm for software development. I could still quickly type in some code — a unit test followed by some production code " and rapidly build up a chunk of functionality. The difference, however, was that I no longer had to spend the next couple of days debugging the code the hard way. I've never looked back.

And the fact I've never looked back might be the very reason I'm overlooking the wisdom of the crowd of anti-TDD activists. Could they be right? Should I stop using TDD in my work? In the interest of keeping an open mind, and cognizant of the wisdom of crowds (50 million flies can't be wrong, after all), I decided to conduct a little experiment to re-validate the value of TDD, one way or the other.

Yesterday, my pairing partner and I picked up a story card to work on. It involved adding some functionality to existing code. I suggested we forego TDD and play the story the old-fashioned way. My partner was skeptical and worried that we were recklessly speeding toward the nearest ditch. We did it anyway. If the anti-TDD crowd is right, then we would achieve at least the same results in at most the same amount of time as we could by using TDD.

Working from the back end toward the UI, we made a small database change, modified a couple of SQL statements here and there in the code, added some logic in a few Java classes, and made a small change to an XHTML document. It all looked pretty good and we felt we hadn't overlooked anything. After 90 minutes of work, we were feeling confident and happy.

Then we started hacking through the debugging and manual testing process that is always necessary when one does not test-drive code. It took us the rest of yesterday and all of today to get to "done." I think it was a fair experiment, because TDD was the only variable we changed. We still did pair programming and continuous integration. The Product Owner and the database specialist were in the team room, available for questions and assistance. Having worked on the project for a while now, we knew that we could have completed a similar task in about two to four hours, working on the same code base, in the same team room, on the same project, with the same people, had we used TDD as usual.

So, 50 million flies can be wrong.

TDD isn't a value-neutral choice. It really is better than other approaches. Those who promote it are not snake-oil salesmen or religious zealots. They just care about quality software and professionalism. Reject "agile" if you wish. Just beware of throwing out the baby with the bathwater.

Disclaimer: No code was harmed in the making of this experiment. We went back and remediated the missing tests.

At the family dinner table the other day, my son asked me, "What do you actually do in your job, dad?"

Me: Basically, most companies are run like this: <pantomime action="shooting myself in the foot"> "Bang! Ow! That hurt! Bang! Ow! That hurt! Bang! Ow! That hurt!" </pantomime> So, they call me in and ask what they should do. I say, "Stop doing that," and they go <pantomime action="holding a gun and not shooting myself in the foot"> "Hey, that's a lot better! Thanks!"</pantomime>

Son: Can you be more specific?

Me: Okay, here's one example. To carry out any given business initiative, people with a range of different skills have to play their respective parts. A business initiative might require marketing people, attorneys, financial analysts, software specialists, and others.

Son: Yeah, that makes sense.

Me: So, most companies organize all the people who have a given skill into the same group, and keep the groups separate from each other. Each business initiative gets chopped up into little tasks, each of which is done by some specialist in one of the separate groups. The groups try to put the initiative together by making formal requests for "services" from one another, using a cumbersome, bureaucratic process, but no one has a clear overarching view of the goal. From the point of view of any one employee, their work looks like an endless series of disconnected, isolated tasks that have no context.

Son: Why don't they just put people with all the skills necessary for the business thing in the same room so they can talk to each other directly?

Me: Well, that's what I recommend. And they go <pantomime action="holding a gun and not shooting myself in the foot"> "Hey, that's a lot better! Thanks!"</pantomime>

Son: It's kind of obvious, isn't it?

Me: Sure. As you've just demonstrated, it's child's play.

Son: Then what do they need you for? Why don't they just do it themselves?

Me: Because most companies are not run by children.

Various memory aids have been concocted to help us remember guidelines for writing good code, like SOLID, GRASP, and KISS. In general, these guidelines tend to encourage high cohesion and low coupling of software modules. When modules are tightly coupled, the code becomes hard to understand, hard to modify, hard to reuse, and fragile (a change in one module causes problems in other modules).

One of the classic examples of tight coupling is cyclic dependency: Each of two modules depends on the other. For example, two C++ classes whose header files include each other are cyclically dependent — each depends on the other. Depending on the characteristics of the programming language, tight coupling may lead to additional challenges, as well. For example, Java applications are organized into logical namespaces called packages. If programmers do not pay attention to coupling in their design, they may inadvertently create cyclic dependencies between two Java packages, even if no two individual Java classes have cyclic dependencies. When two packages have cyclic dependencies, it is not possible to release either package independently of the other; software deployment and release processes become complicated. The software works, but it is difficult to live with; in business terms, its total cost of ownership is higher than necessary.

Most of the software design guidelines people talk about are meant to help us achieve high cohesion and low coupling in one way or another. Sometimes, we can detect possible problems in the code by keeping an eye (or "nose") out for so-called code smells, like the ones listed in this article on Coding Horror. Static code analysis tools such as Sonar can detect structural problems in code. Some development environments may offer protection from common coupling problems; for example, Microsoft .NET development tools can prevent a successful build of an application that has cyclic dependencies.

Another very common form of tight coupling is that one module in a system knows too much about the internal implementation details of other modules. The idea of minimizing this sort of coupling has come to be known as the Law of Demeter: "Only talk to your friends." The name comes from the Demeter Project, whose goal is to advance the science and art of adaptive and aspect-oriented programming. The Law of Demeter was popularized by the influential (in software circles) 1999 book, The Pragmatic Programmer: From Journeyman to Master, by Andy Hunt and Dave Thomas.

Unfortunately, it wasn't popularized enough. Despite the wealth of information available on the subject, it seems that the vast majority of people who write software for a living are either (a) completely unaware of any guidelines for sound software design, or (b) largely misunderstand the guidelines. I recently saw some Java code that looked more-or-less like this (this is a sanitized example, of course):

public Thing getThing(Integer id) {
    return new Beta().getGamma().getDelta().getEpsilon().getOmega().getThing(id);
}

This code exhibits three of the basic code smells listed in the aforementioned article on Coding Horror:

• Message Chains
• Inappropriate Intimacy
• Indecent Exposure

 

The Message Chain is fairly obvious: The call to retrieve an instance of Thing goes six layers deep through the application. A change to any one of the referenced classes could force a change in the client class, contrary to the well-known design guideline known as Single Responsibility Principle, which states that a class should have only one reason to change. This is yet another form of tight coupling.

The message chain also exhibits Inappropriate Intimacy in that the client class has to know something about the internal structure of all the other classes referenced in the message chain. The client class should only have to know that Beta exposes a public API to retrieve a Thing instance; it should not know exactly how Beta goes about obtaining the instance.

Beta, Gamma, Delta, and Epsilon all exhibit Indecent Exposure by allowing clients to call through them to reach a third object. None of these classes can be changed independently of the others. They might as well all be written as a single class, a Big Ball of Mud.

Despite the emergence of development methods that encourage and facilitate well-factored code, and the growth of the Software Craftsmanship movement, the Big Ball of Mud remains the most popular design for software, including greenfield development that has the full benefit of hindsight regarding the bad design approaches of the past.

For fun and relaxation, I attended a few sessions devoted to hands-on programming...or at least sessions that sounded as if they would involve hands-on programming.

Pairing games as intentional practice (Moss Collum and Laura Dean)

I expected the session to be about games designed to bring out good and bad practices in pairing, and I expected to be able to play the games to hone my pairing skills. It started out well, with the facilitators asking participants for their good and bad experiences with pairing. There were some high performers in the room, including Corey Haines and J.B. Rainsberger, and many others who had interesting experiences to share. I expected we would then simulate some of these problems and apply some new techniques to solve them.

No such luck. The entire presentation consisted of the two facilitators talking. There was no hands-on exercise and no "games" as such. The games were simply different approaches or styles of pairing. Even so, some of them look pretty useful as "shovels" we can use to dig ourselves out of particular types of problems, when a pair may be stalled for one reason or another.

They discussed four such "games." Ping Pong is simply the basic ping pong pairing style, in which partner A writes a failing unit test, partner B writes the code to make it pass and writes the next failing unit test, and partner B makes that test pass and writes the third test, etc.

Farsighted Navigator uses the basic driver-navigator style with a twist: The navigator can't speak about anything of larger scope than a single method, and the driver can't write anything other than what the navigator talks about. It may be useful when a pair is unsure what to do next and keeps changing direction, or remains at too high a level of abstraction to get any actual code done.

Socrates is another variation on the driver-navigator style. In this case, the driver is not allowed to respond verbally to anything the navigator says, but can only "answer" by writing code. It's a bit harder to see the value of this, unless perhaps the pair has fallen into a pattern of talking too much about what they should write, instead of just writing something and letting the code tell them what the design should be.

In the No Talking game, neither partner speaks, and they try to maintain a good flow of development while passing the keyboard back and forth quickly. They might use a chess timer to create a sense of urgency about keeping the micro-coding sessions very short. I'm somewhat at a loss to understand what problem this solves. I can see that it might shake things up a bit if team members are getting bored with the routine of work.

I think Moss and Laura have a lot of useful information and experiences to share, and I hope they will consider restructuring this session to make it a bit more engaging. One possibility might be to blend the "games" content with the presentation approach we used in "Effective Pairing: The Good, the Bad, and the Ugly" at Agile 2008 (with Brett Schuchert, Lasse Koskela, and Ryan Hoegg), Agile 2009 (with Brett Schuchert, Lasse Koskela, and George Dinwiddie), and Devoxx 2009. We acted out specific anti-patterns in pairing, and then asked the audience to identify the problems with the interaction and suggest corrective action. The "games" Moss and Laura came up with could be used as specific suggestions to overcome the anti-patterns.

The worst of legacy code: Forensic development (Jason Kerney and Llewellen Falco)

This session wasn't about "agile," but it was still very interesting and practical. Each of the presenters has had the privilege of working in a production support role and dealing with extremely difficult legacy code. When you have to fix a bug in production, you don't have the luxury of time to study and understand the application fully. You need techniques that help you isolate the source of the bug quickly. In this session, the presenters introduced us to two specific techniques: The Peel and The Slice.

The presenters themselves can explain what the session was all about better than I can. They prepared a video and posted it to YouTube, here: http://www.youtube.com/watch?v=yAKL6rlEF_s. The video goes through basically the same presentation as they made in person to introduce the two techniques during the session.

The debugging exercise was based on a snippet of legacy code that one of the presenters actually had to support in a past job (I don't recall which one). The code is available online here: http://pastie.org/843507. The bug report stated that when three of the loans is associated with "Tom," then two records are written to the database for "Tom" instead of one.

The fact the code is in Russian is amusing, but not really relevant to the exercise except that it illustrates an interesting fact about finding production bugs: You don't need to know what the application is supposed to be all about. You just need to find the bit of code that isn't behaving properly, according to the bug report. With that in mind, it can be misleading to read comments, method names, and class names that do not accurately reflect the behavior of the application. If you can't read these things, then they can't mislead you. There's nothing you can do other than run the code and make it reproduce the reported behavior.

The basic idea is to get the code to run so that you can isolate the cause of the error. A chunk of code out of context won't run, of course, because there's no database, no framework, no container, no nothin'. In a time-critical debugging situation, you don't have time to set up a test environment with all the bells and whistles. As the video demonstrates, you can peel away unnecessary code to get at the code you need to run, and/or slice out the code you need to run and omit deeply-buried "pits" that prevent you running it.

To help you, you might use a coverage tool to tell you when you have successfully set up the code to run completely, and a mocking library to help you fake out external dependencies so you can focus on the problematic code. Although mocking libraries are often associated with agile-style development, this is not really a development exercise and has nothing to do with "agile" as such.

The presenters used EasyMock to help them fake out dependencies. When they did so, a "Boo!" was heard from the back of the room, followed by laughter from the group. EasyMock seems to have a bad rep these days. As I watched the presenters step through the exercise, I realized that an advantage of EasyMock in this situation is that it is "strict" by default, but not nearly as cumbersome to set up as JMock2. While most of us would prefer JMock2 when we need a strict mocking framework for new development, the relatively simple EasyMock is a good fit for a debugging situation. We want the strict behavior by default to expose hidden method calls when we are exploring the code (by executing it).

The process consists of getting the code to run somehow, usually by creating a class with a main method and bringing in just enough of the production code to get it running. We expect to get a number of stack traces along the way. That is how we learn about hidden method calls we need to mock. To keep the stack trace relatively free of clutter, it's just as well to run with a main method and dispense with test runners. It's interesting that you can get into a pretty good flow using the Slice and the Peel, and isolate the true cause of the bug in far less time than it would take to try and learn the code base "properly." The code you write for exploration purposes is throw-away code.

Check out the video and the code snippet. If you ever have to debug a production problem, you might find the techniques useful. The examples were done in Java, but the Slice and Peel are techniques and are not language-dependent.

Large-scale refactoring using the Mikado Method (Ola Ellnestam and Daniel Brolund)

In this session, we learned about another useful technique for working with legacy code, and it, too, is not specific to "agile." The Mikado Method is named for a pick-up-sticks game called Mikado. In Mikado, one of the sticks is substantially more valuable than the others. It rarely happens to wind up on top of the pile when you drop the sticks, so you have to work carefully and in a step-by-step manner to reach it. Similarly, when you have to refactor a legacy code base to add a feature, the bits you need to cull out of the Big Ball of Mud are rarely obvious or easy to access.

Mikado takes the approach of beginning with the end in mind. You begin by creating a dependency graph starting with the end goal. Then you identify the immediate prerequisites to reaching the goal, and continue identifying dependencies in that way until you reach a reasonable starting point. What is a reasonable starting point? That will be a judgment call and context-dependent. In any case, once you've got the dependency graph, you start working your way back from the leaves toward the goal, step by step.

Despite the session title, this isn't really a technique for large-scale refactoring. Indeed, you perform only the minimum amount of refactoring necessary to achieve the goal. The problem space for the Mikado Method is the addition of a new feature to an existing code base, especially when the existing code base is not already well-factored.

Here are some online resources about the Mikado Method, mostly written by Ola and Daniel:

• http://mikadomethod.wordpress.com/ — Blog and PDF book
• http://github.com/brolund/mikadomethod — Examples

 

Doing agile right using the right tools (Hadi Hariri)

This was not a programmed session, but a vendor demonstration on behalf of JetBrains. Hadi demonstrated BDD development in .NET using MSpec and Resharper. MSpec is short for "Machine Specifications."

Oddly, I was the only attendee until about halfway through, when we were joined by a second. Not really sure how that happened, except that the demo took place during a lunch break. Lunch breaks were 90 minutes, so there was plenty of time for people to do other things. The Open Jam areas were always busy, anyway.

MSpec is an interesting tool and a useful addition to the .NET toolkit. As Microsoft winds down the IronRuby project, MSpec may become more important since it is unclear how we will be able to use Cucumber for BDD in the .NET environment. The tool looks good, and Hadi's skills as a presenter and demonstrator are excellent. Time well spent. Here are some online resources:

• Article by Aaron Jensen on CodeBetter.com
• machine.specifications on github

 

Nano-incremental development: Elephant carpaccio (Andrew Shafer and Alistair Cockburn)

This was a hands-on coding session in which we practiced breaking requirements down into the thinnest possible slices and delivering them in very short iterations of 9 minutes. We were divided into teams of two or three.

This is an exercise Alistair devised to help programmers understand that it is actually possible to build solutions in very small increments, and then to help them understand how to do so. Alistair describes the exercise in detail on his website, and he encourages people who have gone through it under his guidance to use it in their own organizations.

Given an assignment that was pretty small to begin with, we had to build and demonstrate very small increments of the solution. One member of each team functioned as the customer, and could clarify the requirements in any way he/she saw fit so that the team would understand the relative value of each nano-feature.

Our team consisted of three people. We managed to get through the complete assignment on time, although we might have done a bit more refactoring of the "production" code if we had had the time.

In the debrief, Alistair asked for a show of hands to see how many teams had completed all the requirements. About half the teams raised their hands. He then asked how many teams had used a rigorous test-driven development process, including incremental refactoring of both the production code and the tests, to build the solution. Several teams raised their hands. The intersection of the two groups comprised one member: Our team.

Alistair apparently had expected that no one would complete the full assignment using rigorous TDD, since the assignment was so small that the overhead of the TDD cycle could easily overwhelm a team's ability to deliver anything in just 9 minutes. He also suggested that because of time pressure, agile methods tend to encourage sloppy code. He ignored the fact our team's hands were raised, maybe because it was statistically insignificant, maybe because he just didn't notice, or maybe because we didn't have much time left for discussion.

In any case, I disagree with the idea that agile methods (or any other methods) particularly encourage or discourage sloppy code. I think it's a question of self-discipline and attention to craftsmanship. Our team of three consisted of people who had never met, who represented three different age cohorts, who came from different countries, who represented different genders (only two of those, of course), spoke different languages, had different levels of formal education, and had different professional backgrounds. We collaborated at all times and discussed different alternatives at each step in the exercise. This diverse team was able to apply strict TDD to the problem and complete everything within the short time frame allowed, while maintaining discipline in TDD and code craftsmanship. Even with the very short timeframes used in the exercise, there was time to do all these things properly.

One of the highlights of the conference was the session presented by Arlo Belshee and James Shore, "Bloody Stupid Johnson Teaches Agile." The presenters acted out a play they had written that poked fun at every agile stereotype you've probably heard of. By popular demand they repeated it later in the week. It was recorded and I hope to see it appear online soon.

Using a variety of hats and props, Arlo and James played the roles of several stereotyped characters from the agile world that were easy to recognize: The Consultant, The Salesman, The Bishop, The Champion, The Wizard, The Jester. Their goal: To craft the Perfect Agile Process (PAP).

In a hilarious and beautifully performed play, they called out many commonly-heard criticisms (both valid and otherwise) about agile itself and about those who promote it. I can't even begin to list the jabs at agile thought-leaders and well-known speakers and writers in the community, including the presenters themselves.

We in the audience were advised to call out "PAP! PAP! PAP! PAP!" whenever we heard pap from the stage. To help us understand all about agile, we were given Instructions (cans of Bud Light). The Instructions were warm, but much appreciated.

There were some very useful audience participation activities, too. In one, the group was divided into two teams. The Waterfall team received a set of instructions, and the Agile team received a different set. The Waterfall team was assigned to tear paper into as many small pieces as they possibly could in the time allotted. Meanwhile, the Agile team was assigned to fold as many paper airplanes as possible in the same amount of time. Then, the two teams threw their results at one another. The team whose products flew better was the winner. Agile was clearly more effective than Waterfall.

In another exercise, the two teams are lined up in single-file along opposing walls. The Waterfall team has to organize into functional silos and take a set of instructions through a series of handoffs with quality gates. The Agile team has to repeat the phrase "Yay Agile!" to one another down the line. By the end of the time period, the Agile team has delivered its message to the facilitators, while the Waterfall team is still in the Design Phase. Agile wins again! And it's scientific-like, too.

According to the session proposal, the presenters' goals were to highlight mistakes we often make in implementing agile methods:

• Giving in instead of removing impediments
• Focusing on planning practices to the exclusion of delivery practices
• "One True Way"ism
• Ignoring high-bandwidth communication
• Leaving out the customer
• Not integrating testers
• Process navel gazing
• Directive leadership

I would say they achieved those goals.

 

After the play, they iterated through the characters and polled the audience, asking how many of us had seen the character in our work, and then how many of us had been the character in our work. Then they led a discussion of which characteristics of each character we would want to keep, and which we would want to eliminate. For example, The Bishop keeps us on track by reminding us of fundamentals, but can be dogmatic and lose sight of practical matters. The Wizard answers every question by saying, "It depends." When asked On what? he replies, "It depends on the context." And that's all anyone can get out of him. On the plus side: He recognizes that we can't just drop a cookie-cutter agile process on top of every situation. On the minus side: He can't bring himself to take any action; he's stuck in analysis paralysis. There is a similar dichotomy in each stereotyped character.

One of the best things about the session was that The Salesman teamed up with Bloody Stupid Johnson to create an official certification program. We received printed certificates attesting to our status as Agile Software Specialists. I'm going to frame mine.

I spent quite a bit of my time during the week in this area. I attended four sessions that directly or indirectly addressed the topic, and participated in two informal group discussions about how to improve the state of the art of coaching in the agile space. The coaching-related sessions I attended were very different from one another, and all were presented by people who do this sort of coaching for a living: David Draper, Arto Eskelinen, Sami Honkonen, Rachel Davies, Gino Marckx, and Michael Sahota. It's worth noting that there were many more sessions on the topic of coaching; it was a strong focus of the conference program.

Retrospective for an agile coach (David Draper)

David shared three past experiences in coaching agile teams. He set out a timeline for each, consisting of a horizontal line representing time and a series of boxes highlighting key events in the project. The timeline was general and did not represent absolute increments of time. Each box contained a small smiling or frowning face icon, and some boxes contained one of each.

David summarized the course of events in each case and described what had happened at each of the key points in the timeline. Then, working in small groups, we talked about what had happened and how a coach might have approached each situation differently.

The value of the session was that each of us gained new insights and ideas from our peers that can help us handle similar situations in our own work.

At one point, David described an incident in which the team presented a project roadmap to the business stakeholders that demonstrated the team would have to increase its velocity exponentially (and here I do not use the word in its loose sense) in order to meet the prescribed deadline. The stakeholders did not understand the implication, and told the team to proceed with the plan. David wrapped up the story with a quotable quote: "When you tell people what they desperately want to hear, they don't necessarily detect the fact you're saying it tongue-in-cheeck."

That's a very valuable lesson to take from the story. All too often, we who understand how agile works assume it's perfectly obvious one cannot play games with velocity and force a given scope to fit into a given schedule. It's not so obvious to others, especially when an on-time delivery of the full scope is in their vested interest. We have to lead them by the nose toward understanding.

Effective questions for an agile coach (Arto Eskelinen and Sami Honkonen)

This session was very different in content and in presentation style from David's. In a very well-structured series of short explanations and short workshop activities, Arto and Sami introduced us to a coaching model called GROW: Goal, Reality, Options, and What To Do. They reinforced a concept we should all know, but that we can forget in the heat of the moment: Our role as coaches is to create awareness and a sense of responsibility, and not merely to hand people instant solutions to their problems.

A key point was that the way in which we formulate questions has a great deal to do with whether we will receive meaningful and actionable responses. They pointed out that a good coaching question has the following characteristics:

• It leads to exploration
• It aims for a descriptive answer
• It avoids judgment
• It avoids unproductive states of mind

Generally, these will be What? When? How many? How much? questions. Who? and Why? questions can easily lead to defensiveness and blame-shifting.

 

Arto and Sami suggest that unproductive states of mind can be created by asking questions that cause:

• Laying blame
• Justification
• Denial
• Guilt
• Shame
• Obligation

It's hard not to notice the similarity between this list and the levels of personal responsibility defined in Christopher Avery's Responsibility Process (short of the final level, Responsibility):

• Denial
• Lay Blame
• Justify
• Shame
• Obligation

So, if we can keep in mind basic principles of personal responsibility, we won't go far wrong in formulating constructive questions with coachees.

 

In discussing the session with Arto and Sami before we began, I offered the following negative example: "Did you write that code on purpose, or did you just let your cat walk across the keyboard?" They agreed that this was indeed a negative example. It in no way leads to exploration; it does not guide the person toward a descriptive answer; it is highly judgmental; it creates a defensive state of mind.

I must admit it was not a random example. Not so long ago, I was sitting with a programmer on a team I was coaching and reviewing some of her code. I asked, "Have you heard of the Single Responsibility Principle?" She cheerfully recited a textbook definition of SRP. The code itself did not bespeak a profound understanding of SRP, but at least I had dodged the sarcasm bullet. It was sheer dumb luck that she took the question so literally. In any case, the question did not lead to any improvement of awareness or responsibility.

In the real case, I found that in subsequent interactions with that team member I was able to help her see potential improvements in her code by asking more specific questions about particular sections of the code, and by focusing on questions about how the structure of the code might affect future modification and support of the application. It would have saved us both some time had I participated in this session first.

The coachee might not realize there is a problem at all. In that case, we need questions with the characteristics listed above to help him/her reach an understanding that there is something to be done. In the personal example, the team member did not perceive any problems with her code. A few exploratory questions could have set the stage for a constructive working session.

In the Goal phase, we are trying to find out what problem the coachee wants to solve, and how he/she will recognize success when it happens. We want to help identify a goal that is:

• high enough
• positive
• meaningful
• specific

In applying this advice to the personal case, I find myself at a loss to define a good goal prior to working through at least the Reality and Options portions of the GROW model. A preliminary goal statement might be:

• Carry out a refactoring that takes the class closer to a clean design.

There's no assumption that we can completely refactor the class in one fell swoop, so at least it isn't unrealistic. It sounds positive. On the other hand, it isn't really meaningful, and definitely not specific. We don't have enough information yet to know which of the several responsibilities the class supports would be the best choice for the initial refactoring exercise.

 

At this point, I'm thinking the GROW model doesn't necessarily have to be applied sequentially, although that's the way the workshop exercises introduced it. In retrospecting the real case, I thought that maybe the Reality phase would enable us to craft a better goal statement.

According to the GROW model, good Reality questions are designed to explore the coachee's view on the current state. We want to find questions that:

• test assumptions
• explore different angles
• expose feelings

Relating this to my personal example, it was clear that the team member needed guidance to discover the problems in the structure of her code. I did get around to this, despite my poor opening question. Feelings exposed: Pride of workmanship. Assumptions tested: What does SRP actually look like in real life? Different angles explored: What sorts of changes would improve this code? What sorts of changes are possible? New feelings exposed: Desire for greater pride of workmanship. No feeling of blame, judgment, or obligation. So far, so good; but we still don't have a good goal statement.

 

The Options phase of the GROW model looks for alternate paths to the desired state. To that end, questions should:

• state existing ideas
• challenge limitations
• include discussion of "stupid" ideas
• bring out at least 3 alternatives

I have to pare down the details of the real case considerably in the interest of space. Suffice it to say the class in question was to be part of a webapp generated by a relatively rigid framework. The framework would call methods in this class to execute custom code as part of its built-in request/response cycle. The obvious and crude implementation (and the one recommended in the product tutorial) was to jam all the custom code directly into the callback methods. The problem in real life is this approach results in a bloated class that performs several distinct functions.

 

Relating the Options phase of GROW to the personal case, I could have used well-crafted questions to help expose the team member's existing assumptions about the structure of the class and the possibility of refactoring the code, to challenge the apparent limitations imposed by the rigid framework, explore a few "stupid" approaches in hopes this might spark some out-of-the-box thinking, and bring out 3 or more alternative approaches to improved design.

At this point, we could have returned to the Goal step armed with information obtained from the Reality and Options steps. We would have enough information to craft a goal for a first pass at refactoring the class that was high enough (it wouldn't be easy in any case), positive (it would extract the largest chunk of secondary functionality), meaningful (it would clearly improve the readability of the code, if nothing else), and specific (it would address one particular secondary responsibility).

That takes us to the What To Do step, the W in GROW. Arto and Sami described this as "agreement to walk the path." In the case of my personal example, we could define a specific refactoring that had a clear definition of done, an observable outcome, and a target date. The session was about coaching in general, and not all coaching has anything to do with technical issues. Agreement to walk a path of improvement in mindset, process, or principles may mean that the coachee agrees to begin walking the path, or to resume walking a path from which he/she has strayed. In the more general case, the What? questions should examine:

• What to do?
• When to do it?
• Observable outcome?
• Obstacles removed?
• Uncertainty cleared?

You might think that the GROW model applies best in general coaching situations, and may have less to offer in a technical coaching situation. In fact, technical professionals have very sensitive emotions, especially with respect to their own code. There are a lot of programmers out there who perceive themselves as very advanced, very "senior" professionals who already have a long track record of success. Yet, most of them have no inkling of Software Craftsmanship. We have to be able to explore relevant information and bring out unpleasant facts without touching the wrong buttons. Otherwise, we won't get anywhere.

This was a truly excellent session. If you have a chance to see it on another occasion, don't miss it. For more information, download http://sami.honkonen.fi/checklists.pdf.

Agile Coaching Dojo (Rachel Davies)

Rachel was intrigued by the coding dojo concept and wanted to see whether it could be used to improve coaching skills. This session was a run-through of a dojo structure she created for coaches.

Working in groups of six or seven, we took the roles of seeker, coach, observer, and facilitator. The seeker was the person seeking help. The seeker is a coach who has a challenging coaching situation and seeks advice from his/her peers. Each group had one of these. The coach is just what the name implies. Each group had 2 or 3 coaches, who interacted with the seeker one at a time for five minutes each. The observer did not speak, but observed the nature of the interaction between each coach and the seeker, and provided feedback after all the coaches had taken a turn. The observer's feedback was not coaching advice, but was limited to comments on the interaction itself. The facilitator kept time and kept everyone within the boundaries of their roles.

Everyone chose the role they wanted to play. The first step was for the seekers to choose a coaching problem from their own experience. They discussed it in roundtable fashion with their group. Then, each seeker left their original group and joined a different group, where no one had heard about the problem. Then the kata began. Each coach interviewed the seeker for five minutes, one at a time. Other group members did not speak during the interviews. Observers observed, and made notes if they wished. After the interviews, the observers discussed their observations of the style of interaction with the rest of the group. At the conference, there was insufficient time for multiple rounds, but in an actual dojo everyone would have a chance to play each of the roles at least once.

It seemed a worthwhile exercise to me. The seeker at our table said he came away with some concrete ideas about how to coach the people in his organization through the problem he was having. It reminded me somewhat of a session at Agile 2007 along the same lines. In Coaching 101: A Role-Playing Session, Jeff Nielsen and Dave McMunn led us through contrived coaching situations in groups of three: Coach, coachee, and observer. The session leaders and some of their colleagues acted as facilitators, to answer questions about the process and to keep time. In that session, the facilitators created contrived situations; coaches could just as well use real scenarios. The three-person format may allow for more-focused interaction and faster role changes than the seven-person format.

Another possible advantage of the three-person format is that there may not be enough coaches in any given local area to form even a single group of seven for regular practice. For example, David Draper was in our group, and he mentioned that a coaching dojo would be a great thing for his company, but they have only five agile coaches in the London area. Using the three-person format, his company could organize an internal coaching dojo with the five coaches plus another employee in a role that calls for coaching skills, such as a project manager or Scrum Master. The six could have a very productive 90-minute dojo session with ample opportunity to act in all three roles, and to explore several coaching scenarios from their various clients. Coaches who work in the same general area, but who work for different companies or independently, could also organize their own dojos.

Regardless of the format, repeated practice of the basics is as important for honing coaching skills as it is for honing programming skills. I hope Rachel will continue to refine the idea and move it forward.

Quadrants of Effectiveness Game (Gino Marckx and Michael Sahota)

This is a board game for five participants designed to illustrate strategies for maximizing the effectiveness of a team. It's described on Gino's blog and in this set of presentation slides. After the session, I asked Gino if the game materials were available for purchase, because I want to use the game with my own clients. He said the materials would be available sometime soon.

I won't go into details about game play. The basic idea is that each group of players makes moves based on cards, as in trading card games. Players move markers around a board. Each player has two markers — one that tracks the urgency of the current move, and one that keeps track of the number of points the player has accumulated.

The point of the game is to demonstrate the effect of different strategies for maximizing value. Certain aspects of the scoring are not explained at the outset, so that players will craft individual strategies. At some point, the facilitator explains that the real goal is for each team to maximize its points. The five players at each table constitute a single team, not five individual competitors.

At our table, people played competitively at first. Once the team scoring aspect was explained, the team's dynamic changed to a collaborative style. Using the collaborative strategy, we finished the session with the second-highest score. The highest-scoring team had also changed to a collaborative strategy early in the game.

The results also brought out another aspect of team dynamics: The speed with which decisions are made. We did not count the number of rounds played at each table, but we speculated that the speed of game play may have accounted for the difference in scores between the top two teams. Our team was very slow to make each move. The team members discussed options and possible moves for some time before reaching a decision. We wondered whether this caused us to play fewer rounds in the allotted time, and therefore to accumulate fewer points than the top-scoring team.

The relevance to real team dynamics is that we cannot take too long to make decisions in our projects, because events will overtake us. We have to be able to make decisions quickly based on limited information. That is not an explicit part of the Quadrants of Effectiveness Game; it is simply an observation based on the outcome we obtained in the session.

In general, games are a great way to bring out important points we want our clients to understand so that they can make well-informed decisions during the agile transformation. This particular game is especially powerful and simple enough to learn quickly. I relate this session to the theme of improving coaching skills because I like to use games in my coaching of non-technical people and non-technical practices.

Informal discussions of coaching in the agile space

On Tuesday morning, I stumbled into a great conversation. In search of breakfast in the hotel restaurant, I encountered a group of people waiting for a table. I knew some of them, so I asked if I could join them for breakfast. They warned me they were going to have a breakfast meeting, so I could sit with them as long as I kept my head down and my mouth shut. I had planned to keep my head down anyway so that I could shovel food into my mouth, so I agreed.

This was no random assortment of breakfast-seekers. The instigator was Rachel Weston, Director of Services at Rally Software Development, and the group comprised Lyssa Adkins, author of Coaching Agile Teams: A Companion for ScrumMasters, Agile Coaches, and Project Managers in Transition; Rachel Davies, author of Agile Coaching; Alan Atlas; David Chilcott; Julie Chickering; and Mark Kilby — all agile coaches.

What brought these big brains together was the growing need for coaches in the agile community, and the lack of support for coaches who want to improve their skills or exchange ideas with peers. These individuals, who are leaders within the agile community in the area of coaching coaches, are not necessarily leaders in the broader field of coaching. They were full of ideas, and Rachel Davies agreed to set up a forum on the Agile Alliance website to serve as a collection point for links, contacts, and other information relevant to coaching the coaches.

Unfortunately, I have been unable to access that forum. I had a series of problems getting logged into the site at all, and ultimately gave up. I'm very interested in being part of the coaching-the-coaches initiative, but not interested in websites that prevent people from getting in. I expect eventually there will be another entry point for internet access to these resources. Rachel made it clear that she was willing to get things started by creating the forum, but that she would not be able to carry the load of volunteer work to keep the thing going. I think we will need a user-friendly site that doesn't ask for passwords and annual membership fees.

The group agreed to host a discussion in the Open Space area later in the week. I attended that, and was pleased to see at least twenty professional agile coaches in attendance. The group generated even more good ideas, which were noted on flip chart sheets. I think this initiative will result in a higher level of coaching skill throughout the community.

I attended a couple of sessions aimed at people who usually work in roles different from my own. I wanted to gain a better understanding of how they perceive agile methods and how they see themselves participating in agile projects.

Dance of the Tester (Janet Gregory)

I came to the agile community from the standpoint of a programmer. I had been working in the role of Enterprise Architect at the time our company embarked on an agile journey. I dropped the title to join an XP team in the role of Team Member, and learned a lot about how different types of work are performed in an agile workflow. Although I perform the work of other roles, including tester, analyst, project manager, Scrum Master, technical lead, and team coach, it was informative to learn how an agile implementation looks and feels from a perspective other than that of a programmer.

Janet Gregory's intent was not to provide me with insight into the perspective of a software tester. Dance of the Tester is an introductory overview of the role of tester on an agile project. Although it is an introductory session, I learned a lot from it because it looks at agile work from a perspective other than my own. I filled the back of an evaluation form with specific praise for the session, and could have filled another sheet.

She emphasized certain aspects of agile thinking, including whole team, fast feedback, verbal communication, automation, retrospectives, and (interestingly) discipline. In describing the difference in mindset between a traditional tester and an agile tester, she used a phrase that struck home and that I intend to borrow: constructive skepticism. The tester's goal is to contribute to the whole team's effort to deliver high-quality software, and not merely to look for bugs in the code. A mindset of constructive skepticism keeps the tester's role positive and collaborative. It's a powerful concept.

To help explain how different types of testing occur on an agile project, who typically performs them, and why they are done, Janet used the canonical agile testing quadrants diagram to frame this part of her presentation:

Janet described a comprehensive roadmap for testing activities in an agile context. It spanned project planning, release planning, iteration pre-planning and planning, testers' direct contributions during iterations, system testing, and support for deployment and releasing. It was clear throughout that the information was based on direct, practical experience and any discussion of theory was pinned to field experience.

If you want more from Janet Gregory, check out her website at http://janetgregory.ca/ and her book (with Lisa Crispin), Agile Testing: A Practical Guide for Testers and Agile Teams.

Beyond Staggered Sprints: Agile user experience design teams in the real world (Jeff Gothelf)

Another session that provided great insights into non-programmer perspectives was Jeff Gothelf's "Beyond staggered sprints: Agile user experience design teams in the real world." I was especially interested in this because I'm currently working on a project that is heavily driven by the user experience design, at a client that is implementing agile methods for the first time. Jeff described the experience of the UX design team at his company after the software development group decided to implement agile methods — without consulting his team or anyone else in the company. It was a classic bottom-up agile adoption (and still ongoing).

There was good attendance at the session. I didn't count, but I think there must have been at least 40 people in the room. I didn't recognize anyone at all. That was my first indication that I was in the right place to learn about an unfamiliar perspective.

Jeff described each of four "attempts" to align the UX design team with the agile processes the software group wanted to use. At the outset, the UX team operated as an internal service that did work for several projects concurrently on behalf of four lines of business. This structure posed no problems in the context of the waterfall process they had been using at the company. When the software group suddenly and unexpectedly (from the point of view of the UX team) introduced "agile" methods, the UX team had to adapt without having any previous experience or knowledge of agile, and without the benefit of any training or coaching. Jeff scoured the internet to find that there was a dearth of information focusing on UX designers. He found some value in Jeff Patton's work, and engaged Alistair Cockburn to have a look at the process, but on the whole the UX team was on its own.

First attempt. The software group adopted a two-week iteration length. So, the UX team's first attempt to align with this model was to squeeze the waterfall process into two weeks, complete with all the handoffs, reviews, and quality gates. I trust I need not tell you how well that worked. They also mimicked the software team's use of whiteboards. They put 14 rolling whiteboards in the UX design work space. Jeff used internal surveys to gauge people's response to the changes. One respondent to a survey during this phase of their transition wrote, "The whiteboards do not help organize the UX team's work at all. Instead, they block out natural light from the windows and create a harsh and uncreative visual environment."

The UX team also stopped doing functional specs in advance. They did not understand how to compensate for this in an agile process, so they began to use the cards on the whiteboards to capture the functional specs. The boards had multiple purposes and were cluttered with all kinds of information. The information that was no longer being provided by functional specs also bled over into the wireframes. They came to be heavily annotated with all sorts of details that normally are not included in wireframes. This was a direct consequence of their being told that functional specs aren't "agile," while no alternative was offered.

The UX design team produced a large chart to explain how they felt about the whole thing. Their perspective was aptly summarized by a box on the chart labeled, "Agile = project never complete." As far as they could tell, agile work amounted to an endless series of small tasks. There was no endpoint, no celebrations, no larger context. Some of the other boxes on the chart (and there were many) read as follows:

• Negative environment that fosters failure and generates low morale
• Best experience never gets built
• Deadlines before information
• Conjured sense of urgency
• No insight into larger goals
• No ideation time - skills not being used fully
• Minimum viable product (?)
• "Going for the bronze!"

My main take-away from this is that all those who will be affected by a change of process have to be engaged during the planning stages for implementing the new process. All these problems occurred because the UX design team was blind-sided by the sudden shift to agile methods. Under the heading of Misery Loves Company, I can say that people in other roles also experience these feelings during the first few months of agile adoption.

 

Second attempt. The UX design team crafted a Style Guide to provide people with standard information about design elements, so that every project would have have to start from Square One. It contains color palettes, font standards, HTML forms standards, and many other details. They also started to build a library of reusable UI assets; widgets already customized according to the Style Guide. These innovations drastically reduced the amount of time needed to generate UI elements for projects. It also opened the door for people other than UX designers to pick up design tasks, depending on how the workload looked at any given time.

And the downside...

• "Everybody's a designer" — zombies?
• Cycles valued over creativity

That's a much shorter list of downsides than we saw from the first attempt, but these are qualitatively serious downsides. Why would a designer continue to work in such an environment at all?

 

Looking at it from a programmer's perspective, I can say that we like to abstract the mundane, repetitive stuff out of our way so that we can focus on the more-interesting problem solving aspects of our work. We use reference architectures, design patterns, third-party libraries, frameworks, templates, UX Style Guides, and anything else that helps us avoid repeatedly creating the same code over and over again. What is left over is the real brainwork — the reason we entered the field in the first place.

When we do the same thing with UX design work, what is left over after everything has been abstracted away? UX designers want to create the first design of a New Thing. They want to create an experience, not just another cookie-cutter website. When all they have to do is make selections from a predefined library of assets, where's the creativity? What's the point?

Third attempt. The UX design team was feeling disconnected from the whole process. They put everything in-line; that is, the UX designers are now part of specific cross-functional Scrum teams, each dedicated to one line of business. They carry out usability testing every other week; they bring in real (external) customers for this. The UX team incorporates feedback from these sessions into their current and future projects. Things got better.

Fourth attempt (ongoing). Jeff described this as bringing everyone together and then pulling everyone apart. Before starting the routine iterations of a new project, everyone involved — not only the UX design team — participates in a design session. They do collaborative sketching in which they generate 6 ideas in 5 minutes. They then take 5 minutes to boil it down to 3 or 4 ideas. Another session results in a single idea. They keep all the sketches for future reference.

Jeff showed us examples of sketches that were created during these sessions. He asked the group to identify the primary role of the person who drew each sketch. It was easy to see which were created by programmers, managers, and so forth as opposed to "real" designers. They are finding value in this deep level of collaboration, even if most people aren't trained artists.

This is where the company stands as of the day of the presentation. It has taken them over a year to get to a place where the UX design team does not feel excluded and discouraged. Jeff said that the programmers are now asking the UX team to size their tasks in terms of story points. They will try that very soon. Personally, I'm not too enthusiastic about that. The nature of the work is very different. There is no reason to assume both should use the same techniques to manage their tasks. But time will tell.

Personally, I wanted to ask Jeff what the original business drivers had been to try agile methods in the first place. Had it been a response to business problems that had been measured and assessed, or had it just been something the technical staff was keen to try? I didn't get a chance to ask him directly. What raises the question in my mind is the fact the UX design team had no idea such a profound change was about to happen in the company.

Needless to say, every session and every conversation throughout the week added to my understanding of how people perceive agile these days, and where they think we should be going. There were quite a few informal discussions of the topic every day. For me, the following contributed strongly to this area of learning:

• Alistair Cockburn's Oath of Non-Allegiance
• A new certification program, the International Consortium for Agile
• Session: Whither Agile? A participant-driven round table (me)
• Session: Today's best practice is tomorrow's process smell (me)

The Oath is simply a statement that we will consider any idea regardless of its source, rather than limiting ourselves to some predefined and branded package of ideas. It came about as a reaction to all the pointless bickering in the community the past few months, and to the tendency for agile proponents to judge everything according to whether it fits a predefined model of "agile" regardless of whether it adds value in a given situation.

 

The ICA is a community response to the circular debate about certification, and about who gets to decide how to certify people, and so forth. Companies can join the ICA, but it is not controlled by any one corporation or organization. It is a work in progress, but already has a pretty well-designed framework for a certification program.

The two sessions listed were both mine, so I can safely summarize their intent. I had hoped the round table discussion would help us come around to a consistent view of where "agile" stands as of today and in which direction we, as a community, want to take it next. The best practice/process smell session was meant to help us identify cases when we have become stuck using a suboptimal process because we've had to work around a tactical issue, and then never got around to solving the root cause. Neither session turned out as I had expected.

Let me review the round table session first. If we take the goals of agreement on a common vision and selection of a clear direction as the acceptance criteria for the session, then it failed. Fortunately, the unexpected (to me, anyway) outcome of the discussion provided valuable learnings (to me, anyway).

The discussion was very open-ended and was largely participant-driven, and yet I did have a few ideas I wanted to float to the group. I tried to relate the present state of agile to the canonical innovation adoption curve and to map out a continuum from "heavyweight" to "lightweight" processes. I was prepared to introduce a few other angles on the topic, too, but these two along with ideas the participants brought with them were more than enough to fill the three-hour time slot. Several people stayed afterwards to continue the discussion on their own.

I wrote "2000" and "2010" on two sticky index cards, and applied them to a large drawing of the innovation adoption curve. I placed the 2000 card at the edge of the Chasm, and the 2010 card near the end of the Early Adoption phase. This represented my concept of the penetration of "agile" at those points in time. After a moment of silence (one of the few, I'm happy to say), one of the participants walked up to the wall and repositioned the 2010 card at the top of the curve, mid-way through the Early Adoption phase. Heads nodded in agreement.

I talked through an analogy I had intended to demonstrate, but I never purchased the food coloring. Picture a large container of clear water. That's the IT industry circa 2000. Now picture a small vial of dark blue fluid (it was to be water with blue food coloring). The agile movement introduced the dark blue water into the IT industry — pour the contents of the small vial into the larger container. Ten years later, the IT industry is pale blue. It will never be dark blue, as the proponents of agile envision success, but it is definitely more blue than it was before. This is a sort of success. The question is, where do we go from here? We aren't at the same state we were in 2000, so the same solutions probably won't work. How do things differ today?

Some participants related to the analogy, but most disagreed with the premise. They felt it was possible to make the water darker, and we should not accept poor implementations of agile as "successes;" not even partial ones. One participant took the idea in a different direction. Laurent Bossavit noted that if we began with milk and added a drop of something, that a living culture would begin to grow in the milk and the result would be cheese. One does not debate cheese with a Frenchman. Beyond that, I found his analogy powerful. The colored water is the result of planting seeds that never grow; the cheese is the result of beginning a process that takes hold and moves forward on its own. (I also like the play on the word "culture," but that's neither here nor there.)

Laurent's take on the growth process provided a segue to another notion I wanted to explore with the group. My understanding is that the people who wrote the Agile Manifesto originally called their meeting the lightweight process summit. They were interested in removing waste from IT processes, to transform them from "heavyweight" to "lightweight." Apparently, some members of that group feared that people would misunderstand "lightweight" to mean "not robust." They decided to look for another word. They found one.

I wanted to get back to the fundamental idea of making our processes lighter. The agile community has benefited from an infusion of ideas from Lean Thinking in the past few years. Key lean ideas pertaining to the idea of lighter processes include:

• Perfection — always keep perfection in mind as a target, even though we understand perfection is not actually achievable;
• Continuous improvement — use the Five Focusing Steps from the Theory of Constraints as a framework for improving our processes, moving them closer to perfection; and
• Focus first on value, second on continuous flow, and third on removing waste from our processes.

To start this part of the discussion, I posted a large drawing of a horizontal line labeled "heavyweight" on the left and "lightweight" on the right. I had prepared several index cards with the names of various approaches, models, or practices in the areas of organizational structure, process framework, and development practices. I held up each card, the group reached enough of a consensus about word meanings to prevent the discussion from stalling, and we collectively decided where on the spectrum the item belonged.

 

These were, for example, things like "Linear SDLC process," "Time-boxed iterative process," and "Flow-based process," in the category of process frameworks, along with things like "Time-based task estimation," "Relative sizing of work items," and "Single-piece pull w/o task estimation" in the category of practices. There were other categories, too. We laid these things out along the line and associated items that were prerequisite to others.

The idea was to map out a continuum of agile practice from heavy to light, with the assumption that our target is to make things as light as possible. At first, the group bunched most of the items in the middle of the spectrum. They protested that "we can't do away with practice ABC because of constraint XYZ." True. They protested that a linear process can be lightweight and an "agile" process can be heavyweight, depending on how one does things. True. One participant pointed out that single-piece pull may simply be inappropriate in the context of software development. Possibly true, especially in the absence of a long list of prerequisites that are barely even on the horizon of general management and IT practice today. They were focusing on the as possible part instead of on the as light part of the statement.

These are present-day-reality truths; they are not inherent attributes of the models and practices on the chart. I tried to guide the discussion toward distinguishing between the inherent "lightness" of each item on the one hand, and the practical reality of implementing that item in participants' current circumstances. In that light, the group gradually agreed to spread the cards out a little more, so that we could see a sort of progression from heavy to light. My goal in this was to get us a little closer to an idea of "perfection" (in the lean sense) that we could use as a reference for deciding whether an opportunity for improvement exists in our work environment. Without that sort of reference, how can we know whether improvement is possible or what sort of change would be helpful?

This seemed to open the sluices for fresh ideas. Everyone in the room had been thinking about this issue for some time already, and they all had great ideas. We brought up some of the better-known emerging ideas such as Capabilities-Based Planning, Real Options, Beyond Budgeting, and restructuring organizations such that application development and support functions reside directly in the business units they serve. We also questioned some of the Lean concepts that are being talked about in agile circles these days. In particular, the whole concept of the value stream was questioned; another notion, the value network, was proposed by one participant. The same participant questioned whether Throughput Accounting was a good fit for software development. (Sorry, I've forgotten his name; if you know who it was, please post a comment or send me an email. He works for Emergn, if that helps jog the memory.)

My second session was entitled, "Today's best practice is tomorrow's process smell." I made some foolish assumptions in preparing this session. The first was that I assumed an "expert" level session would draw participants who were already applying agile principles and practices very well, and who had reached an impasse or a challenge of some sort in moving ahead with continuous improvement. That assumption led to other assumptions about how much context the participants would need in order to contribute to the discussion. I prepared insufficient physical tokens of the content, such as charts, cards, handouts, or slides, and provided insufficient introduction to provide context for the discussion.

I had hoped participants would bring examples of work-arounds or "painkillers" they had implemented in their own organizations, and that had become institutionalized, thus preventing them from removing the root cause problems that had led to the work-arounds in the first place. I had provided a few examples in the session proposal as well as on my blog to give people a sense of what I had in mind. It wasn't sufficient to set expectations for the session. To distinguish between a "good idea" and a "work-around," a person has to have a concept about heavy-to-light process improvement similar to what I introduced in my first session.

I learned that the majority of people who believe they are already practicing "agile" do not have a deep understanding of agile values and principles, and simply cannot judge how effective their current practices are. They can see that whatever they are doing today in the name of "agile" is working much better than whatever they were doing before, but they don't really understand why. They are simply following a defined process that was taught them by a consultant or that they read about in a book. They cannot tell whether anything they are doing on their projects is a work-around as opposed to a good practice; their assessment of any given practice is not based on whether it adds value or incurs overhead, but merely on whether it matches the defined process.

Rather than painkillers or work-arounds, participants described issues with agile implementation that are common in the early months of an organization's initial experiments with agile methods, when only a few people in the organization are interested in change, and the change agents have no past experience with the new methods. Most of their problems are organizational impediments, and not problems with agile methods as such. In describing their problems, participants did not explain how they had crafted a work-around to a specific problem, and how the work-around had become institutionalized. Instead, they simply stated a problem and waited for me to hand them a solution; and the problems were of massive scope. This sort of thing was far afield of the intent of the session.

I'm afraid these participants did not come away from the session with much useful information. I can say that I learned from them, though. I gained a clearer idea of how agile is being implemented in the industry, and how well-understood the concepts really are. Going back to the blue water analogy, I would say the water is very, very pale blue. Agile penetration is wide, but not deep.

Another informative angle for me was that a few participants believed (and still believe, I guess) that what they are currently doing in their organizations represents not just a "good" agile environment, but an exemplary one that requires no improvement at all. Usually, their descriptions of the current state in their organizations is horrifying.

In one case, a participant who holds a Ph.D. in agile methods described a nightmarish 1980s-style dysfunctional organization that uses a pseudo-RUP process in a heavily-siloed organizational structure, decorated with every sort of high-ceremony, high-overhead gates and checks and controls you can imagine. This in itself is not a problem, provided the current state is recognized as a problem and the change agents in the organization are working toward continuous improvement. In this case, if I take his description literally, then the organization settled into this mode of work some six years ago, and has not changed anything since. The most puzzling bit was that he seemed to be proud of it all. I must say I admire his perseverance, at least. In his place, I think I would have long since traded my laptop for a push-cart, and turned to selling popcorn on the street.

Since these were my sessions, I'll review the feedback I received, and how I intend to use the feedback. Feedback often falls into one of two patterns: Reviews are bunched in the middle, or reviews are bunched on each end (U-shaped reviews). Both these sessions garnered U-shaped reviews. For the round table, all the formal review forms that were submitted were strongly positive. However, about half the people who started the session decided not to return after the break. I take that as a negative review (they voted with their feet), although it does not offer me any information to improve the session in future. For the process smells session, the formal reviews included both very positive feedback and very negative feedback that included concrete recommendations for improvement, and nothing in the middle. Thanks to everyone who took the time to provide the latter. I think the process smells topic deserves more attention, and I will revise this presentation to include better contextual information.

If you were following the Agile 2010 conference on Twitter last week, then you probably know more about it than I do. They had a live Twitter feed running on several large-screen monitors, but I didn't read them and didn't go online very often during the week. I wanted to be fully present and immersed in the experience, so I probably missed a lot. FWIW, here are my impressions.

With approximately 1,400 participants, this year's conference was slightly smaller than the last two or three, but still quite large. I was happy to participate in several excellent sessions, but disappointed that I had to miss a few, since there were two or more compelling choices in every time slot. At the same time, there were a lot of very smart people whom I was happy to meet and listen to between sessions, at mealtimes, in the Open Jam areas, and at conference- or vendor-sponsored social mixers. Most of the "big names" whose proposed sessions were not accepted for the program attended the conference anyway, and were very active in other people's sessions and in informal settings throughout the week. So, is large size a good thing or a bad thing for a conference? Yes.

I had no particular plan in mind at the start of the conference. In hindsight, it looks as if my conference experience tended to fall into a handful of categories:

• Learning about other people's perspectives about the current state and future direction of agile adoption and the agile community;
• Learning how agile adoption can affect people who work in roles or disciplines other than my own;
• Learning about effective coaching and how to grow as a coach; and
• Learning about techniques and tools for developing code, refactoring, and debugging legacy code.

I'm going to break this up into separate blog posts because of the length.

 

The secret to writing good automated tests is to be a good software designer.

It sounds trivial - but the sad fact is that more often than not the attempt to implement test automation falls due to poor software design where people are struggling with strong dependencies where you cannot test a single class or module without also configuring a whole environment of databases, webservices, populating whatever tables etc.

When the novice experiences this the natural, but incorrect reaction, is to write fewer and fewer tests at a higher and higher level of integration, the limit case being manual end-to-end integration testing through the GUI.

It is a symptom of bad software design, not automated testing being worthless.

In a good project the test suite does not need to be changed when external data changes. In a bad project, the test suite breaks when time passes or every time someone runs the end-of-month job to add interest to the accounts, or when the ticker price changes for a stock.

In a good project, only a few localized changes to tests and production code are needed as for a requirement change. In a bad project, there is impact analysis and lots of similar changes all over the code-base to accomplish this.

In a good project most of the code does not know from where it gets its data - in a bad project, even a simple interest calculation business rule knows about how to look up the database and the interest rate web service.

In a good project there is typically 1-2x as much test code as production code. In a bad project there is often 5x-25x as much production code as test code.

In a good project most tests exercise only a single class or a few functions. In a bad project every tests activates a code-path through many parts and tiers of the application.

In a good project most of the test code is asserting that things went as expected. In a bad project, most of the test code is setting up the environment to run the test.

In a good project there are tests at all levels of the application. In a bad project they are mostly through the UI and often even manual.

In a good project you can change the application without risk. In a bad project change is painful, infrequent, risky and requires executive sign-off.

In a good project you can deploy new code to production safely many times a day. In a bad project you only do it reluctantly every monthly or a few times a year.

On Tuesday next week, I'll be facilitating a collaborative session to explore the phenomenon of addiction to pain-killers, as it occurs in software development organizations.

A pain-killer relieves pain for a short while, but does not treat the underlying condition that caused the pain in the first place. When the pain returns, the course of least resistance is to take more of the pain-killer. The patient becomes conditioned to the pain-killer and requires progressively stronger doses to relieve the pain. The patient runs the risk of developing a dependency on the pain-killer, while the underlying condition remains untreated.

Software development organizations experience "pain," too: The pain of delivery delays, missed targets, high defect rates, Death Marches, misunderstood requirements, and many other issues. In this context, a "pain-killer" is a work-around to a problem that is causing delays or other difficulties in delivery. It is a tactical solution, usually addressing the observable symptoms of a deeper problem.

Since a work-around exists, people don't bother to look for the original problem. They become careless, comfortable in the knowledge that a formal, well-defined work-around will take care of any symptoms that may arise. This is equivalent to the body becoming conditioned to a pain-killer, and needing ever-stronger doses to relieve the pain.

As the work-around is repeated and refined, the organization can become dependent on it. The organization may create formal job descriptions for people who specialize in the work-around. It becomes more and more difficult to dislodge the work-around from the organization's culture and procedures. When people hold retrospectives or plan Kaizen events, they focus on improving their implementation of the work-around rather than on identifying and eliminating the original problem.

People cease to think of the work-around as a work-around; they cease to perceive the problem as a problem. In effect, the organization institutionalizes the underlying problem rather than solving it. With employee turnover, new hires are indoctrinated to believe the work-around is a necessary and important part of the organization's culture and procedures. As older employees leave, their memory of the origin of the work-around is lost to the organization. "That's the way it has to be done."

Eventually, the specialists responsible for the work-around may form a professional organization of their own and start to hold conferences, publish trade journals, and present awards to one another. Other companies see all this and rush to implement the same underlying problems in their own organizations, so that they can begin to practice the successful work-around, too. "We'd better learn to do this, too, because that's the way it has to be done."

To bring this little rant back around to the overall theme of the Agile 2010 conference, remember that the Snowbird meeting in 2001 was originally called the "lightweight process summit." Its participants were interested in identifying the key characteristics of software development and delivery processes that tended to yield good results, and to eliminate as much of the waste from such processes as was practical.

In those days, the mainstream processes for software development and delivery were "heavyweight" — characterized by cost overruns, schedule overruns, long lead times, functional silos, indirect means of communication, hand-offs, large batch sizes, massive work-in-process inventories, and a debilitating lack of alignment with the consumers of the software. The authors of the Agile Manifesto were looking for ways to make those processes "light" by trimming the fat.

The idea of keeping things light and focusing on the delivery of value is fundamental to agile thinking. It wasn't invented with agile thinking, but it is fundamental to the agile approach to IT work. When organizations re-introduce fat into their processes as a way to alleviate short-term pain, they take a step on the slipperly slope back to the Old Ways. That's why the topic is relevant to this conference.

In the previously-referenced article, you can read about two obvious examples of this: The extended User Story, which becomes more and more like an old-style Requirements Specification Document, and the Mini-Waterfall-Within-Each-Iteration anti-pattern. A colleague on my current project mentioned another example that he has experienced on more than one team: A daily team meeting (not a stand-up) in which team members warn one another about which parts of the code base each developer touched during the day. The red-flag question in that one is this: Why don't team members already know the status of the code base, without the need for a special meeting about it?

A few additional examples are noted in the session proposal, the text of which is reproduced below. It is my hope that participants will bring many more examples to the table, as well.

Session proposal

The premise: A “best practice” at one level of maturity becomes a “process smell” that guides us to the next level of maturity. There is a tendency to “lock in” a set of assumptions and practices and to assume this represents “agile best practice.” Rather than locking in, we should be guided by the principle, “question everything.” This applies especially to our own assumptions about what constitutes agile best practice. Otherwise, our thinking will ossify and we will cease to improve the overall agile toolkit and our own ability to add value for customers.

The session begins with a brief introductory talk to describe the premise and seed the discussion. After that, the remaining time is given over to participants to self-organize and collectively lead an examination of various “good” or “best” agile practices that are typically assumed to be de rigeur for agile environments. Depending on the number of participants, we may split up into small groups and then have each group present its findings at the end of the time slot.

The speaker wonders: Do we really “question everything,” or have we settled on certain assumptions about how things must be? Do we question (for instance):

• Organizational structure
• End-to-end delivery process
• Budgeting
• Recruiting and hiring
• Incentives and rewards
• Practices and techniques
• Traditional roles
• Habitual language
• Conventional career paths

Examples of widely-used good practices we may wish to examine in a new light:

Practice: Sashimi technique for organizing user stories into sizes that fit into an iteration.
Value: Addresses the problem of fitting variable-size user stories into fixed-length iterations.
Smell: Seems as if the goal of this work is merely to fit stories into time-boxes. Customers don’t buy time-boxes. Therefore, this is muda - non-value-add activity.
Questions leading to the next level: Is there a way to deliver value without this activity? Why do we need to fit the work into fixed-length iterations? Would it be a smoother work flow if we just worked the stories in priority order, regardless of size? What would that approach imply about upstream preparation of the work queue, backlog, or portfolio? What must we do to enable the organization to handle the lighter-weight approach?
Categories: End-to-end delivery process, practices and techniques

Practice: HR checklist to help HR screen candidates and to give teams some warning and preparation when a new team member is to be added.
Value: Addresses the disruption caused when new arrivals are assigned to a team by management without warning and without the team’s consent.
Smell: Agile principles of “stable team” and “self-organizing team” do not appear to be honored. Seems “reactive” rather than “proactive” - assumes management can and should dump new people onto teams helter-skelter. Ignores effect of changing team composition on team dynamics; fails to acknowledge that velocity or throughput will be reduced (temporarily) with each occurrence.
Questions leading to the next level: Why isn’t the team itself recruiting and auditioning new members? What is the appropriate role for HR?
Categories: Organizational structure, recruiting and hiring, traditional roles, habitual language

Practice: Planning Poker, story sizing, assorted improved estimation techniques
Value: Better than traditional time-based task estimation for short-term (iteration) planning and for release planning.
Smell: Estimation is not customer-defined value. Time spent doing estimation is time lost to value-add work. High-level estimation at project or program scope is valuable; low-level estimation of MMFs, User Stories, or technical tasks is largely useless.
Questions leading to the next level: Is there a way to do short-term planning and keep the work flowing without bothering to do small-scale story- or task-level estimation? What are the dependencies of taking this approach to short-term planning? How can we take care of those dependencies?
Categories: End-to-end delivery process

Practice: Customer collaboration
Value: Addresses organizational issues such as lack of feedback, poor communication, low trust, features out of line with needs, long lead times, rework, and misunderstandings.
Smell: The word “collaboration” implies cooperation between two or more distinct entities that are working toward a common tactical goal, but that are otherwise separate. The implicit assumption is that the entities must or should be separate, with bridges for communication and cooperation under the heading of “collaboration.”
Questions leading to the next level: Is there a way to achieve seamless integration rather than merely “alignment” and “collaboration” among all interested parties? Can we re-think old assumptions about organizational structure, functional silos, professional career paths, and roles in a way that would eliminate the need for explicit “collaboration” and make collaborative work natural?
Categories: Organizational structure, end-to-end delivery process, traditional roles, conventional career paths

Practice: Agile portfolio management
Value: Alleviates problems inherent in traditional portfolio management, such as the pursuit of unnecessary projects; duplication of effort; misalignment of effort with strategic business plan; budget approval on a different schedule than project and program planning, resulting in “scurrying” toward the end of the annual budget cycle to “use up” funds so that we can qualify for the same or a greater budget next year, without reference to the business plan.
Smell: The existence of an “IT portfolio” indicates a disconnect between the central IT department and the “business side of the house.”
Questions leading to the next level: Why do we need a portfolio at all? Why do we need discrete projects at all? What prevents us from treating business ideas as Real Options? How can we manage budgeting on a just-in-time basis?
Categories: Organizational structure, end-to-end delivery process, budgeting, habitual language

Learning outcomes

• Refresh the healthy (and fundamentally agile) habit of questioning everything, including our own assumptions and our habitual methods and techniques.
• Examine several examples of widely-applied good agile practices to understand the specific problems they are designed to solve, and consider whether locking in those practices might prevent us from taking the next logical steps on the path of continuous improvement.
• Return to the real world with a renewed ability and willingness to examine our most comfortable assumptions with a fresh eye.

 

That's the end of the session proposal. On a side note, I will be facilitating a session on the topic, "The IT Portfolio as a Form of Waste," at the Lean & Kanban 2010 conference in Antwerp this September. It starts with the idea that the very existence of an IT portfolio is a work-around for some other source of pain, as a starting point for looking at the whole problem of corporate IT services through the lens of Lean thinking.

I would like to post a link to that conference's website, but unfortunately it will not display for me. They have (too) cleverly included code to see which browser I'm using, and refuses to let me in unless and until I "upgrade" to the latest version of one of the browsers they prefer. Their code is faulty, because I'm already using the latest version of one of those browsers. I suspect their code only works for Microsoft Windows and, possibly, Mac OS X. I'm running on Linux. Unfortunately, again, the site (too) cleverly prevents me even from submitting an email to inquire about it.

It's interesting that the official website of a conference dedicated to improving the value of software to its users has such a glaring, ridiculous, and unacceptable bug. It just goes to show that we still have a lot of work to do to bring IT work to true professional status, and that the place to start is inside the organizations founded and staffed by the (ostensible) thought-leaders in the field. They ought to practice what they preach, even if no one else does.

I'll be facilitating a discussion at Agile 2010 next Monday concerning the current state and future direction of the "agile" community/movement. The session is titled, "Whither Agile? A Participant-Driven Round Table."

After some ten years of activity in the IT industry, things are very different than they were when the Agile Manifesto was written. In my opinion, the time has come for the agile community to reconsider and update its goals and purpose in light of these changes.

Consider the innovation adoption curve presented by Everett Rogers in his 1962 book, Diffusion of Innovations, as refined by Geoffrey Moore in Crossing the Chasm (1991 and 1999). The image is from Wikipedia.

The ideas expressed in the Agile Manifesto quickly took hold, and many individuals and companies took them forward in different ways. Circa 2002, the agile approach stood at the brink of the "chasm," where it might or might not become a part of mainstream IT thinking and practice. By 2008, the good news for proponents of agile methods was that the ideas had, indeed, crossed the chasm.

And that was the bad news, too.

With agile ideas and methods now beyond the peak of the early majority adoption phase, the situation is very different than it was when a small group of enthusiasts shared a common vision as they promoted a well-defined conceptual framework for improving the state of the art in software development and delivery. Today, "agile" has been absorbed into the fabric of IT thinking and practice. It is no longer an innovation — everybody does it (whatever "it" is).

As "agile" enters the late majority adoption phase, the ideas and methods have been diffused into the general methods of IT organizations. There is no single, narrow definition of "agile." The definition has expanded along with the variations in practice that have emerged over the years. A very broad range of ideas, methods, and techniques may be deemed consistent with the definition of "agile."

A consulting firm that targets the late majority adopters with a branded package of agile practices will emphasize predictability, repeatability, and governance; values that hardly seem innovative. At the same time, a leading-edge innovator who pushes the envelope of project management and software development practices, constantly looking for ways to improve delivery effectiveness, may move away from canonical agile practices for completely different reasons, and in completely different ways. Neither of these follows conventional agile methods anymore, and each has taken the ideas in a different direction; yet, both can credibly claim a place in the agile community.

Once an innovation passes the peak of the early majority adoption phase, it is no longer an "innovation" in the true sense of the word. It may not yet have been adopted by everyone, but by that stage it is hardly a cutting-edge idea. The fact laggards haven't caught up with the mainstream doesn't make old ideas new again. One of the characteristics of the agile community today is that many of its members still see their mission as one of introducing innovation; and they are still introducing the same innovation as they were ten years ago. It is as if they have remained stationary while the world has changed around them.

Consider this representation of the Satir change model. I snagged it from Dale Emery's site because it doesn't have a copyright notice. (Don't tell Dale.)

In 2000, ideas, methods, and practices like Scrum and Extreme Programming represented a foreign element that promised to address many of the endemic problems in the IT industry, even before the buzzword "agile" was coined. Agile as defined in the 2001 meeting at the Snowbird lodge targeted those problems well; it targeted the old status quo in IT as it existed at the time. To many who were frustrated with the bureaucracy that had grown up around them in the IT field, agile was a breath of fresh air. (I myself was about to leave the IT profession altogether when I discovered agile, and found it alleviated those problems effectively enough to keep me interested in IT work.)

As agile passed beyond the peak of the early majority adoption phase of the innovation model, it became the new status quo in the Satir model. As agile progressed down the far side of the early majority adoption phase toward the late majority adoption phase, it replaced the old status quo. The status quo in IT today is not what it was ten years ago. Should it surprise anyone that ten-year-old solutions don't seem very innovative?

The aforementioned stereotypical consulting firm (of which there are many real-world examples) understands that as agile moves into the late majority adoption phase, it is time to stop innovating and start milking the cash cow. This is not "wrong" or "anti-agile." It's just good business.

At the same time, the innovators among us are dissatisfied and frustrated with repeating the same first-generation agile material. Most of the people they encounter in industry are already familiar with the basics, and are already practicing some subset of agile methods. The innovators are keen to move forward; to put into practice the notion of continuous improvement. They want to identify the next foreign element, and continue to work at the level of innovators and early adopters.

So, the question comes down to this: How do we wish to define ourselves going forward? Is there still a need for an "agile community?" If so, on which aspects of this expanding and evolving set of ideas and practices does the community wish to focus? Do we wish to embrace the full range, from innovators to cash cow milkers and everything in between? Which emerging ideas and trends are moving into the early majority adoption phase as of 2010, and do we want to consider any of those in redefining or clarifying our mission? Does the word "agile" still serve our needs? Is there more to change than just a word?

Here's the full text of the proposal I submitted to the conference.

Proposed session

When a breakthrough idea crosses the chasm it passes through a make-or-break stage of maturation. Without leadership, even the best ideas can evaporate into nothingness or ossify into dogmatism. Agile stands at just such a turning point now. The message has become fragmented, diluted, co-opted for profit, mischaracterized, abused, and attached as a label to just about anything. What can the community do to see that the proven value of agile principles and practices is not lost in the general chaos or surrendered to commercial interests that have a branded, packaged solution to sell?

“Agile” has been a very useful word. It has served as a point of focus for thought leadership and as a rallying-cry for those who were interested in improving the state of the art in IT and who had been pursuing disparate paths toward improvement. The effects on industry have been profound and far-reaching.

Today, the word “agile” is all but meaningless. The fundamental concepts have been absorbed into the everyday working assumptions of nearly all IT organizations. Like any other transforming idea that is widely adopted, “agile” itself has been transformed by the many people who have applied it to different situations. With each new adaptation, the word loses some of its distinct meaning and becomes integrated into the general fabric of IT work. It becomes less necessary, less powerful, less useful because there is less and less difference between agile thinking and the status quo.

That is the definition of success for any transformative movement in any field of endeavor.

One indication this has been true for some time already is the fact that Forrester has finally gotten around to noticing it. See http://tinyurl.com/y8zvksv.

As I see it, the agile community today has the following characteristics:

• Tacit and near-universal avoidance of questioning anything that might be controversial, such as changing organizational structure, eliminating obsolete specialized roles, or growing beyond the need for the sacred cows of first-generation agile
• Addiction to “pain-killers” - that is, work-arounds to organizational problems that begin as temporary expedients and eventually become treated as “necessary” aspects of an agile work flow; we try to get better at doing the workaround instead of coming back to the original root cause that led to the workaround in the first place
• Allowing commercial interests to take control of the definition of “agile” to match whatever they happen to be selling
• Confusing ends and means - Instead of remembering the practical concerns that led us to change the way IT work is done, we ask “How agile are you?” and set goals such as “Be Agile,” as if agile itself were the end rather than a means to an end
• Balkanization into camps that promote one or another branded packaging of agile ideas and practices, and that snipe at other camps
• Coming at problems in the spirit of one who has a hammer and sees nails everywhere

 

I have been asked to be more specific about what I would like the outcome of this discussion to be. I don’t know. I don’t have the answers. That is the whole reason to suggest a discussion. I don’t even know whether there will be any interest in this discussion in the agile community; I’m afraid I might be questioning things that are too close to the heart. But I’m more afraid of shying away from such questions.

In my view, there are three broad areas in which IT work was completely dysfunctional throughout the 1980s and 1990s, and problems in these areas gave rise to various attempts at improvement, including agile:

• Process issues - The mechanics of taking ideas from concept to cash.
• Human issues - Job satisfaction, commitment, motivation, enjoyment, and work-life balance.
• Technique or (as we would say it today) software craftsmanship - doing work that teaches us something and in which we can take pride.

 

We have lost sight of these areas of focus (with some notable exceptions) and have become distracted by the attempt to perfect specific activities for their own sake. This is not a proposal for 90 minutes of congratulating ourselves. I think we need to figure out which way we want to take “agile” thinking from this point forward…if anywhere.

Learning outcomes

• Identify trends and issues currently threatening to derail or discredit agile adoption
• Identify behaviors within the agile community that may exacerbate these issues
• Identify conflicts within the community, discover common ground, and brainstorm approaches to resolving them
• Begin a broader dialog regarding desirable future directions in growing and directing the agile movement

That was the session proposal. If you want a voice in defining what comes next, join us in the discussion next week. I hope we will at least get started with redefining the community.

How would you like to be able to SMS or tweet your Excel data-crunching code?

With Docjure, it is possible:

(->> (load-workbook "spreadsheet.xlsx")
     (select-sheet "Price List")
     (select-columns {:A :name, :B :price}))

This reads the A and B columns from the Price List sheet in the spreadsheet.xlsx file into a list of maps where the a column has key :name and the B column has key :price.

I believe that is pretty good for code small enough to fit in an SMS.

Exporting your business data to spreadsheets for further analysis is just as easy:

;; Create a spreadsheet and save it
(let [wb (create-workbook "Price List"
                          [["Name" "Price"]
                           ["Foo Widget" 100]
                           ["Bar Widget" 200]])
      sheet (select-sheet "Price List" wb)
      header-row (first (row-seq sheet))]
  (do
    (set-row-style! header-row (create-cell-style! wb {:background :yellow,
                                                       :font {:bold true}}))
    (save-workbook! "spreadsheet.xlsx" wb)))

In just a few lines of code and you have exported the price list to a worksheet, complete with a yellow-background bold header line for extra style points.

In our business applications, bridging to Excel has provided huge benefits:

• Users love it - having their data in Excel enables them to do much more than a static report that can only be changed by a software developer.
• Developers love it - It eliminates a lot of tedious code for generating bespoke reports as we can easily export data into an Excel report template.
• Project managers love it - using spreadsheets provides an easy to understand data exchange format and the flexibility to change reporting features quickly late in the project.
• Sponsors love it - the project saves a lot of time and reduces training cost by leveraging an application the users already know.

As an inspiration, here are some of the things we have used it for:

• Use Excel to calculate currency trading strategies - the traders calculate their currency trading strategies in Excel then import it into the trading system. They benefit from the flexibility of a powerful tool they already know and the trading system takes care of the technical details of the actual trades.
• Exporting to Excel for bespoke analysis - would you like to check your currency trading record? Export all the information to Excel and the traders can slice-and-dice it easily with little or no technical assitance. This is much easier than setting up a reporting database for them to link to.
• Using Excel sheets for content management to facilitate translation - translating an application to all the languages of an international enterprise was easy. Rather than using complex technical XML-files we made Excel sheets the data format for the application strings and had the subsidiaries add their translations to that. Then we put an adapter on the application to convert back and forth to XML-config files and RESX files used internally in the web and Silverlight parts of the application. The translators had a great experience, they had a familiar tool with spell checking and it reduced waste in the translation process by presenting a well-known easily accessible format.
• Exporting to Excel for reporting - set up a spreadsheet template with a data sheet and some reports based on that, then populate it with data from your application. This allows the users to easily change or add reports with no change to the software application itself, thus dramatically reducing the turn-around time for new reports. 

We believe this is so great that we just have to share it:

Therefore, Docjure is free and open-source.

It is written in Clojure for the JVM. Get it at our GitHub page here: http://github.com/ative/docjure and be sure to follow our updates on Twitter: @ativedk (http://twitter.com/ativedk)

It's a new month, and now there is a new Scrumy feature for Pro users: The Scrumy API. Pretty much anyone who has asked us if we have an API recently has already been directed to that page and has been able to access it, but now we're sharing our secrets with the world.

For the uninitiated, an API is an interface that we give to you in order to access the data that we've stored for you in a convenient way. Essentially, it allows you to write your own programs that interact with your Scrumy projects. If, for example, you wanted a big red button that moves all your unfinished tasks into the 'Done' column, you could build that yourself with a few clever API calls.

The Scrumy API is divided into two separate parts: REST and Webhooks.

The REST API allows you to get data from your projects in XML or JSON form using simple URLs. You can also manipulate your data by POSTing or PUTing data to those URLs. You can read all about it at the REST API documentation page.

Webhooks are very different. A Webhook is a URL for an application that you have running on your own server which receives data from us. This means that any time you create or change a task, for example, we will send a piece of data representing the change on your project to that URL. A simple thing you could do with this would be to send a tweet any time you finish a task. Read more at the Webhooks documentation page. Also, the demo is set up to use webhooks, but it works a bit differently than your projects. The demo will allow you to enter 5 webhooks, but none of them will be active for more than 5 minutes. So, if you just want to see how webhooks work, feel free to use the demo, but unless you want to be a jerk, use an empty slot. Then you have 5 minutes to test your heart out.

So those are the big updates for now. If you find errors while reading the apidocs or feel that you could clarify something, feel free to update the documentation. It's a wiki for a reason. If you have any other questions or comments, feel free to contact us at support@scrumy.com.

In response to the floods that ravaged Nashville, Tenn., the Agile2010 conference has been relocated to the Walt Disney World Swan and Dolphin hotel in Orlando, Florida. The conference dates will remain the same; scheduled for Aug. 9 – 13 (http://agile2010.agilealliance.org).

With the uncertainty of when the Nashville facilities could be restored we decided to relocate the Agile2010 conference. For details and answers to questions please click here.

I wanted to provide an update on the schedule for the next few weeks. The program committee, stage producers and reviewers are currently reviewing the 900+ proposals. I know that many of you are wondering when you will hear whether or not your session has been accepted. Here is the schedule for the completion of the program:

• March 3rd – March 26th – Review Submissions and Choose Sessions
• March 29th – April 11th Notifications will be sent to all submitters. The sessions that have been selected the speakers must confirm that they will present the session at the conference when notified.
• April 12th – Program Session Selection completed
• Week of April 26 – Program Content will be scheduled with specific times and rooms
• May 3 – Program Published to web-site

If you have questions or concerns please do not hesitate to let me know.