Skip to content

Feed aggregator

Agile Education Engine - Bringing Power to your Agile Transformation

When we look across the numerous Agile deployment efforts, we see a lower rate of success that we expect.  May I suggest that in order to improve the odds of achieving Agile success and gaining the business results it can bring, there are three success factors. The first is that the Agile change must be thought of a organizational level change.  The second is that the Agile change must focus on getting the mind Agile-ready.  This is emphasized in the article Are you Ready for you Agile Journey.  And the third is that in order to bridge the gap between the Agile values and principles to Agile methods and practices, people need to be well educated in ways to build more customer value, optimize the flow of work, and increase quality with feedback loops.    
The current level of Agile education tends to be limited to 2-days of training and a variety of books.  I think most people will acknowledge that 2 days of Agile training does not provide enough learning.  On the other hand, reading lots of books takes a lot of time and are often not aligned with each other.  The other challenge with some of the Agile education is that it is often focused on implementing the mechanics.
What is missing from many Agile transformations is an Agile Education Engine that helps you truly understand and embrace Agile and helps bridge the gap between Agile Values and Principles and the Agile methods and practices.  This will help folks better understand how to understand, embrace, and implement Agile and move beyond simply following the Agile mechanics of the methods and practices.  The Agile Education Engine can help ready the mind for an effective transformation.  


One of the best Agile education engines that I have seen is the material found in the Value Flow Quality(VFQ) work-based education system.  VFQ provides a set of well-researched topics that are easily digested in the form of readings, case studies, activities, and experiments.  It provides students with the ability to study a little, then experiment a little within their own context (aka, project or team).  The benefit of the VFQ work-based learning system is that it helps people apply their newly learned skills on the job when they need them.  This bodes very well for the learners because, they can learn at their own pace and as they are trying to implement the Agile values, principles, and practices. 
Some topics that VFQ offers are: Why Change, Optimizing Flow, Feedback, Requirements, Priorization, Trade-offs, Understanding your Customer, Delivering Early and Often, Teams, Motivation, Attacking your Queues, Work in Progress, Trade-offs, and more.  Each topic includes a number of techniques that will help you achieve the business outcomes you are looking for.  The VFQ materials will provide you with knowledge on Value Stream Mapping, Story Mapping, Cost of Delay, 6 Prisms and much more.   
In addition, VFQ really helps you get to the state of “being Agile”.  It moves you away from thinking about the mechanics.  Instead, it provides you a layer of knowledge to ensure you apply the principles and behaviors to the practices to gain the outcomes that you want.  Finally, applying the VFQ education is also an excellent way to kick-start an Agile transformation.  This way, the Agile champions for the transformation and teams are armed with a variety of different ways to bring Agile to the organization. 
If you find yourself struggling with getting a good baseline of Agile education, then consider the Value Flow Quality (VFQ) work-based learning system.  It will help you bridge the gap between the Agile values and principles and the mechanics of many of the Agile methods so that you bring the Agile mindset to bear as you start or continue your Agile journey. 
Categories: Blogs

R: ggmap – Overlay shapefile with filled polygon of regions

Mark Needham - Mon, 11/17/2014 - 02:53

I’ve been playing around with plotting maps in R over the last week and got to the point where I wanted to have a google map in the background with a filled polygon on a shapefile in the foreground.

The first bit is reasonably simple – we can just import the ggmap library and make a call to get_map:

> library(ggmap)
> sfMap = map = get_map(location = 'San Francisco', zoom = 12)
2014 11 17 00 27 11

Next I wanted to show the outlines of the different San Francisco zip codes and came across a blog post by Paul Bidanset on Baltimore neighbourhoods which I was able to adapt.

I downloaded a shapefile of San Francisco’s zip codes from the DataSF website and then loaded it into R using the readOGR and spTransform functions from the rgdal package:

> library(rgdal)
> library(ggplot2)
> sfn = readOGR(".","sfzipcodes") %>% spTransform(CRS("+proj=longlat +datum=WGS84"))
> ggplot(data = sfn, aes(x = long, y = lat, group = group)) + geom_path()
2014 11 17 00 38 32

sfn is a spatial type of data frame…

> class(sfn)
[1] "SpatialPolygonsDataFrame"
attr(,"package")
[1] "sp"

…but we need a normal data frame to be able to easily merge other data onto the map and then plot it. We can use ggplot2’s fortify command to do this:

> names(sfn)
[1] "OBJECTID" "ZIP_CODE" "ID"   
 
> sfn.f = sfn %>% fortify(region = 'ZIP_CODE')
 
SFNeighbourhoods  = merge(sfn.f, sfn@data, by.x = 'id', by.y = 'ZIP_CODE')

I then made up some fake values for each zip code so that we could have different colour shadings for each zip code on the visualisation:

> library(dplyr) 
 
> postcodes = SFNeighbourhoods %>% select(id) %>% distinct()
 
> values = data.frame(id = c(postcodes),
                      value = c(runif(postcodes %>% count() %>% unlist(),5.0, 25.0)))

I then merged those values onto SFNeighbourhoods:

> sf = merge(SFNeighbourhoods, values, by.x='id')
 
> sf %>% group_by(id) %>% do(head(., 1)) %>% head(10)
Source: local data frame [10 x 10]
Groups: id
 
      id      long      lat order  hole piece   group OBJECTID    ID     value
1  94102 -122.4193 37.77515     1 FALSE     1 94102.1       14 94102  6.184814
2  94103 -122.4039 37.77006   106 FALSE     1 94103.1       12 94103 21.659752
3  94104 -122.4001 37.79030   255 FALSE     1 94104.1       10 94104  5.173199
4  94105 -122.3925 37.79377   293 FALSE     1 94105.1        2 94105 15.723456
5  94107 -122.4012 37.78202   504 FALSE     1 94107.1        1 94107  8.402726
6  94108 -122.4042 37.79169  2232 FALSE     1 94108.1       11 94108  8.632652
7  94109 -122.4139 37.79046  2304 FALSE     1 94109.1        8 94109 20.129402
8  94110 -122.4217 37.73181  2794 FALSE     1 94110.1       16 94110 12.410610
9  94111 -122.4001 37.79369  3067 FALSE     1 94111.1        9 94111 10.185054
10 94112 -122.4278 37.73469  3334 FALSE     1 94112.1       18 94112 24.297588

Now we can easily plot those colours onto our shapefile by calling geom_polgon instead of geom_path:

> ggplot(sf, aes(long, lat, group = group)) + 
    geom_polygon(aes(fill = value))

2014 11 17 00 49 11

And finally let’s wire it up to our google map:

> ggmap(sfMap) + 
    geom_polygon(aes(fill = value, x = long, y = lat, group = group), 
                 data = sf,
                 alpha = 0.8, 
                 color = "black",
                 size = 0.2)
2014 11 17 00 50 13

I spent way too long with the alpha value set to ‘0’ on this last plot wondering why I wasn’t seeing any shading so don’t make that mistake!

Categories: Blogs

Spark: Parse CSV file and group by column value

Mark Needham - Mon, 11/17/2014 - 00:53

I’ve found myself working with large CSV files quite frequently and realising that my existing toolset didn’t let me explore them quickly I thought I’d spend a bit of time looking at Spark to see if it could help.

I’m working with a crime data set released by the City of Chicago: it’s 1GB in size and contains details of 4 million crimes:

$ ls -alh ~/Downloads/Crimes_-_2001_to_present.csv
-rw-r--r--@ 1 markneedham  staff   1.0G 16 Nov 12:14 /Users/markneedham/Downloads/Crimes_-_2001_to_present.csv
 
$ wc -l ~/Downloads/Crimes_-_2001_to_present.csv
 4193441 /Users/markneedham/Downloads/Crimes_-_2001_to_present.csv

We can get a rough idea of the contents of the file by looking at the first row along with the header:

$ head -n 2 ~/Downloads/Crimes_-_2001_to_present.csv
ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location
9464711,HX114160,01/14/2014 05:00:00 AM,028XX E 80TH ST,0560,ASSAULT,SIMPLE,APARTMENT,false,true,0422,004,7,46,08A,1196652,1852516,2014,01/20/2014 12:40:05 AM,41.75017626412204,-87.55494559131228,"(41.75017626412204, -87.55494559131228)"

I wanted to do a count of the ‘Primary Type’ column to see how many of each crime we have. Using just Unix command line tools this is how we’d do that:

$ time tail +2 ~/Downloads/Crimes_-_2001_to_present.csv | cut -d, -f6  | sort | uniq -c | sort -rn
859197 THEFT
757530 BATTERY
489528 NARCOTICS
488209 CRIMINAL DAMAGE
257310 BURGLARY
253964 OTHER OFFENSE
247386 ASSAULT
197404 MOTOR VEHICLE THEFT
157706 ROBBERY
137538 DECEPTIVE PRACTICE
124974 CRIMINAL TRESPASS
47245 PROSTITUTION
40361 WEAPONS VIOLATION
31585 PUBLIC PEACE VIOLATION
26524 OFFENSE INVOLVING CHILDREN
14788 CRIM SEXUAL ASSAULT
14283 SEX OFFENSE
10632 GAMBLING
8847 LIQUOR LAW VIOLATION
6443 ARSON
5178 INTERFERE WITH PUBLIC OFFICER
4846 HOMICIDE
3585 KIDNAPPING
3147 INTERFERENCE WITH PUBLIC OFFICER
2471 INTIMIDATION
1985 STALKING
 355 OFFENSES INVOLVING CHILDREN
 219 OBSCENITY
  86 PUBLIC INDECENCY
  80 OTHER NARCOTIC VIOLATION
  12 RITUALISM
  12 NON-CRIMINAL
   6 OTHER OFFENSE
   2 NON-CRIMINAL (SUBJECT SPECIFIED)
   2 NON - CRIMINAL
 
real	2m37.495s
user	3m0.337s
sys	0m1.471s

This isn’t too bad but it seems like the type of calculation that Spark is made for so I had a look at how I could go about doing that. To start with I created an SBT project with the following build file:

name := "playground"
 
version := "1.0"
 
scalaVersion := "2.10.4"
 
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
 
libraryDependencies += "net.sf.opencsv" % "opencsv" % "2.3"
 
ideaExcludeFolders += ".idea"
 
ideaExcludeFolders += ".idea_modules"

I downloaded Spark and after unpacking it launched the Spark shell:

$ pwd
/Users/markneedham/projects/spark-play/spark-1.1.0/spark-1.1.0-bin-hadoop1
 
$ ./bin/spark-shell
...
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.1.0
      /_/
 
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_51)
...
Spark context available as sc.
 
scala>

I first import some classes I’m going to need:

scala> import au.com.bytecode.opencsv.CSVParser
import au.com.bytecode.opencsv.CSVParser
 
scala> import org.apache.spark.rdd.RDD
import org.apache.spark.rdd.RDD

Now, following the quick start example, we’ll create a Resilient Distributed Dataset (RDD) from our Crime CSV file:

scala> val crimeFile = "/Users/markneedham/Downloads/Crimes_-_2001_to_present.csv"
crimeFile: String = /Users/markneedham/Downloads/Crimes_-_2001_to_present.csv
 
scala> val crimeData = sc.textFile(crimeFile).cache()
14/11/16 22:31:16 INFO MemoryStore: ensureFreeSpace(32768) called with curMem=0, maxMem=278302556
14/11/16 22:31:16 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 32.0 KB, free 265.4 MB)
crimeData: org.apache.spark.rdd.RDD[String] = /Users/markneedham/Downloads/Crimes_-_2001_to_present.csv MappedRDD[1] at textFile at <console>:17

Our next step is to process each line of the file using our CSV Parser. A simple way to do this would be to create a new CSVParser for each line:

scala> crimeData.map(line => {
         val parser = new CSVParser(',')
         parser.parseLine(line).mkString(",")
       }).take(5).foreach(println)
14/11/16 22:35:49 INFO SparkContext: Starting job: take at <console>:23
...
4/11/16 22:35:49 INFO SparkContext: Job finished: take at <console>:23, took 0.013904 s
ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location
9464711,HX114160,01/14/2014 05:00:00 AM,028XX E 80TH ST,0560,ASSAULT,SIMPLE,APARTMENT,false,true,0422,004,7,46,08A,1196652,1852516,2014,01/20/2014 12:40:05 AM,41.75017626412204,-87.55494559131228,(41.75017626412204, -87.55494559131228)
9460704,HX113741,01/14/2014 04:55:00 AM,091XX S JEFFERY AVE,031A,ROBBERY,ARMED: HANDGUN,SIDEWALK,false,false,0413,004,8,48,03,1191060,1844959,2014,01/18/2014 12:39:56 AM,41.729576153145636,-87.57568059471686,(41.729576153145636, -87.57568059471686)
9460339,HX113740,01/14/2014 04:44:00 AM,040XX W MAYPOLE AVE,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE,false,true,1114,011,28,26,14,1149075,1901099,2014,01/16/2014 12:40:00 AM,41.884543798701515,-87.72803579358926,(41.884543798701515, -87.72803579358926)
9461467,HX114463,01/14/2014 04:43:00 AM,059XX S CICERO AVE,0820,THEFT,$500 AND UNDER,PARKING LOT/GARAGE(NON.RESID.),false,false,0813,008,13,64,06,1145661,1865031,2014,01/16/2014 12:40:00 AM,41.785633535413176,-87.74148516669783,(41.785633535413176, -87.74148516669783)

That works but it’s a bit wasteful to create a new CSVParser each time so instead let’s just create one for each partition that Spark splits our file up into:

scala> crimeData.mapPartitions(lines => {
         val parser = new CSVParser(',')
         lines.map(line => {
           parser.parseLine(line).mkString(",")
         })
       }).take(5).foreach(println)
14/11/16 22:38:44 INFO SparkContext: Starting job: take at <console>:25
...
14/11/16 22:38:44 INFO SparkContext: Job finished: take at <console>:25, took 0.015216 s
ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location
9464711,HX114160,01/14/2014 05:00:00 AM,028XX E 80TH ST,0560,ASSAULT,SIMPLE,APARTMENT,false,true,0422,004,7,46,08A,1196652,1852516,2014,01/20/2014 12:40:05 AM,41.75017626412204,-87.55494559131228,(41.75017626412204, -87.55494559131228)
9460704,HX113741,01/14/2014 04:55:00 AM,091XX S JEFFERY AVE,031A,ROBBERY,ARMED: HANDGUN,SIDEWALK,false,false,0413,004,8,48,03,1191060,1844959,2014,01/18/2014 12:39:56 AM,41.729576153145636,-87.57568059471686,(41.729576153145636, -87.57568059471686)
9460339,HX113740,01/14/2014 04:44:00 AM,040XX W MAYPOLE AVE,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE,false,true,1114,011,28,26,14,1149075,1901099,2014,01/16/2014 12:40:00 AM,41.884543798701515,-87.72803579358926,(41.884543798701515, -87.72803579358926)
9461467,HX114463,01/14/2014 04:43:00 AM,059XX S CICERO AVE,0820,THEFT,$500 AND UNDER,PARKING LOT/GARAGE(NON.RESID.),false,false,0813,008,13,64,06,1145661,1865031,2014,01/16/2014 12:40:00 AM,41.785633535413176,-87.74148516669783,(41.785633535413176, -87.74148516669783)

You’ll notice that we’ve still got the header being printed which isn’t ideal – let’s get rid of it!

I expected there to be a ‘drop’ function which would allow me to do that but in fact there isn’t. Instead we can make use of our knowledge that the first partition will contain the first line and strip it out that way:

scala> def dropHeader(data: RDD[String]): RDD[String] = {
         data.mapPartitionsWithIndex((idx, lines) => {
           if (idx == 0) {
             lines.drop(1)
           }
           lines
         })
       }
dropHeader: (data: org.apache.spark.rdd.RDD[String])org.apache.spark.rdd.RDD[String]

Now let’s grab the first 5 lines again and print them out:

scala> val withoutHeader: RDD[String] = dropHeader(crimeData)
withoutHeader: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[7] at mapPartitionsWithIndex at <console>:14
 
scala> withoutHeader.mapPartitions(lines => {
         val parser = new CSVParser(',')
         lines.map(line => {
           parser.parseLine(line).mkString(",")
         })
       }).take(5).foreach(println)
14/11/16 22:43:27 INFO SparkContext: Starting job: take at <console>:29
...
14/11/16 22:43:27 INFO SparkContext: Job finished: take at <console>:29, took 0.018557 s
9464711,HX114160,01/14/2014 05:00:00 AM,028XX E 80TH ST,0560,ASSAULT,SIMPLE,APARTMENT,false,true,0422,004,7,46,08A,1196652,1852516,2014,01/20/2014 12:40:05 AM,41.75017626412204,-87.55494559131228,(41.75017626412204, -87.55494559131228)
9460704,HX113741,01/14/2014 04:55:00 AM,091XX S JEFFERY AVE,031A,ROBBERY,ARMED: HANDGUN,SIDEWALK,false,false,0413,004,8,48,03,1191060,1844959,2014,01/18/2014 12:39:56 AM,41.729576153145636,-87.57568059471686,(41.729576153145636, -87.57568059471686)
9460339,HX113740,01/14/2014 04:44:00 AM,040XX W MAYPOLE AVE,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE,false,true,1114,011,28,26,14,1149075,1901099,2014,01/16/2014 12:40:00 AM,41.884543798701515,-87.72803579358926,(41.884543798701515, -87.72803579358926)
9461467,HX114463,01/14/2014 04:43:00 AM,059XX S CICERO AVE,0820,THEFT,$500 AND UNDER,PARKING LOT/GARAGE(NON.RESID.),false,false,0813,008,13,64,06,1145661,1865031,2014,01/16/2014 12:40:00 AM,41.785633535413176,-87.74148516669783,(41.785633535413176, -87.74148516669783)
9460355,HX113738,01/14/2014 04:21:00 AM,070XX S PEORIA ST,0820,THEFT,$500 AND UNDER,STREET,true,false,0733,007,17,68,06,1171480,1858195,2014,01/16/2014 12:40:00 AM,41.766348042591375,-87.64702037047671,(41.766348042591375, -87.64702037047671)

We’re finally in good shape to extract the values from the ‘Primary Type’ column and count how many times each of those appears in our data set:

scala> withoutHeader.mapPartitions(lines => {
         val parser=new CSVParser(',')
         lines.map(line => {
           val columns = parser.parseLine(line)
           Array(columns(5)).mkString(",")
         })
       }).countByValue().toList.sortBy(-_._2).foreach(println)
14/11/16 22:45:20 INFO SparkContext: Starting job: countByValue at <console>:30
14/11/16 22:45:20 INFO DAGScheduler: Got job 7 (countByValue at <console>:30) with 32 output partitions (allowLocal=false)
...
14/11/16 22:45:30 INFO SparkContext: Job finished: countByValue at <console>:30, took 9.796565 s
(THEFT,859197)
(BATTERY,757530)
(NARCOTICS,489528)
(CRIMINAL DAMAGE,488209)
(BURGLARY,257310)
(OTHER OFFENSE,253964)
(ASSAULT,247386)
(MOTOR VEHICLE THEFT,197404)
(ROBBERY,157706)
(DECEPTIVE PRACTICE,137538)
(CRIMINAL TRESPASS,124974)
(PROSTITUTION,47245)
(WEAPONS VIOLATION,40361)
(PUBLIC PEACE VIOLATION,31585)
(OFFENSE INVOLVING CHILDREN,26524)
(CRIM SEXUAL ASSAULT,14788)
(SEX OFFENSE,14283)
(GAMBLING,10632)
(LIQUOR LAW VIOLATION,8847)
(ARSON,6443)
(INTERFERE WITH PUBLIC OFFICER,5178)
(HOMICIDE,4846)
(KIDNAPPING,3585)
(INTERFERENCE WITH PUBLIC OFFICER,3147)
(INTIMIDATION,2471)
(STALKING,1985)
(OFFENSES INVOLVING CHILDREN,355)
(OBSCENITY,219)
(PUBLIC INDECENCY,86)
(OTHER NARCOTIC VIOLATION,80)
(NON-CRIMINAL,12)
(RITUALISM,12)
(OTHER OFFENSE ,6)
(NON-CRIMINAL (SUBJECT SPECIFIED),2)
(NON - CRIMINAL,2)

We get the same results as with the Unix commands except it took less than 10 seconds to calculate which is pretty cool!

Categories: Blogs

Agile Testing Days 2014 - Go Wilde In A Safe Place

Agile Thinks and Things - Oana Juncu - Sun, 11/16/2014 - 21:29
The best way to cope with the Post Ecstatic Stress Disorder of a great event as Agile Testing Days 2014 is to write a blog post about.  After a great week where we attempted to test how far our creativity can go and how this creativity can turn in concrete reality  fast, here  is my therapy :)The Legend Of  Table 2Photo Credits to Mike BowlerAgile testing Days conference starts with Speakers Dinner . It's at the end of Tutorials day and before the conference. This year the Speaker Dinner turned epic.
 Agile community is a cheerful tribe.As all real communities it gathers people that share same values and same interests. No need to display them in the lobby. Now, this mutual recognition of the tribe  went a little step further at our table. For those that have heard "table 2!" during the conference and didn't get a grasp of what was happening here is the ( one and only ) true story:
We were cheerful and noisy at our table and everything seem to go smoothly for us until, someone from the table nearby , might  have been Huib Shoots, shouted :
"Shut-up, Table 2 !"
Our table took the challenge and immediately ignited the culture of #Table2 , supported with a long list of toasts. It took about 30 minutes to make  Table 2  "known" and about 40 minutes before the its members decided to propagate the "Table 2 Mindset" to other dinner tables. Which we did .
Beyond a shared joke, there is some deep learning I popped from this super fun event :
Turning a challenge into an opportunity fills  that opportunity with  the value of overcoming the challenge. Some smart assumptions were hold in the "Shut Up , table 2 !",  like "We are  table 1 ,  you're just the next one" ;) . Turned out into an advantage .
The power of purpose creates identity. While truly diverse in terms of focus of interest,  we are all very much on the same page at Agile Testing Days . Throwing in the middle of the group a purpose , makes that group self-organise to meet that purpose with joy and engagement.


The Car in the Lobby


"This is a serious sh..." , says #scrumcat . I'll tell more in a 2nd version of the post. Hosted by Joe Justice and Christine Hegarty from scrumInc , I have a lot to tell about, as it went wildly awesome. If you have read the first version of this post , I promised a longer description of this 4 iteration ( 4 hours ) car building in the lobby of  a smart hotel !







For me the genius of this "happening" lies in the  "It's real life" side of the workshop.  Examples :
  • Parts delivery is late !
  • User stories are big and dependent , team morale is low as it cannot deliver at the end of the sprint.
  • Product Owner is a blocking point because either she cannot answer team's questions either she is assigned to multi-teams and becomes a bottleneck. 
  • Urgent events arrive and have to be addressed in expedite mode
  • Multi-teams product effect : teams may create some troubleshooting for other teams flow of work

Some of our incidents that happened to us and we arrived to address may ring a bell, hmm ?

Proven Evidence of Passed Acceptance criteria of the following User Story : "Urgent, we need more beer"
I enjoyed fully the  car building as a true experiment to prove Agile is not exclusively about software development and related metaphors.  Neither is Testing First a nerd caprice.
The happy after-noon of eXtreme Manufacturing proved that entertainment is a source of performance. Christine and Joe are masters of flooding the space with joy. And fun , fun, fun .The Keynotes
I said it last year. Agile Testing Days has 3 keynotes per day, this is the highest density of keynotes I've seen in a conference . Maybe José Diaz knows that 3 keynotes a day is as healthy for a conference as having three meals a day for an human.  It worked for me. Sitting in a big room to listen to 45 min of inspiring talk just after the meals ( or before the Beer Time) punches up the energy in the space.
Some personal highlights of the keynotes :
Purple version of the future in Lisa's and Janet's keynote about the future of testing was a great example of how trouble shootings ( of the projector) are turn into strengths. I loved how un "hardly to look at" display of slides turned in an awesome testing metaphor #StarTek .
Walking KeynoteWalking Stand-up keynote moment proposed by Fanny and Alexander .  Their keynote was about the mindset of happy change agents. They really demonstrated how happily a crowd of hundreds can turn around a room if they have inspiring change agents to tell them to do so.
Personal story told by Antony Marcano that proves that points out that we are humans , not job description. And as humans we do things, that might be given different activity names. As a Storytelling addict , I deeply enjoyed Antony's well crafted story about his life as a tester . And so much more beyond : a son, a father,  a professional enthusiast. By the way , support Damani to reach for his dream. Start by following him on Twitter ! 
Entertainment is important and Agile applies everywhere , highlighted Joe Justice in his keynote about extreme manufacturing using Scrum . Some simple examples , because bla, bla, bla may not convince ? 
  • Boeing, John Deer and Lockheed Martin build planes and other big hard stuff using eXtreme Manufacturing. 
  • A high-school in Alphen, Netherlands, uses Scrum to build its courses , where pupils organises in teams and teachers are POs
  • The Scrum Index fund : yeah, let's talk big money, not Agile cheesy :  banks invest in companies doing Scrum. Everyone of you not happy  about why Scrum and not Agile as a whole , I invite you to take it from here and make it evolve further. In a very #Table 2 way ;) 



Whatever Happens is an Opportunity To Take a Step Forward
The things went wild at Agile Testing Days 2014. Would you expect Brazilian Dancers and building true cars in the lobby of a at a business event ? If you think "this is not serious" , I can state that there is nothing more serious that creating a safe space where because everything is exceptional , it becomes "regular" . And people explore ideas together in a brave way. You may.
So my takeaway from Agile testing Days 2014 is :

When things go surprisingly unexpected , serendipity happens, and it's up to us to turn opportunities in real options for the future.

And, last and not least, after a very good prototype at Agile 2014, looking forward for the kids track next year.

The Lean Karaoke Definitely Vagif Abilof is a talented for singing  the testing . Hope to provide you with his  ABBA featured "The tester finds it All" singing closing keynote.


Thank you Dear Organizers :
José Diaz, Uwe Gelfert , Madleine Griep


And of course , thanks to the #octoawesome teams that were in my Demo Driven Development workshop :)




Categories: Blogs

The problems with "large, complex projects" in the enterprise

What is a "large, complex project"?When someone says "large and complex" in an enterprise context, they're typically referring to a few attributes:
  • Multiple teams
  • Multiple components, including non-IT components
  • Multiple vendors
  • Distributed teams
  • Year plus timelines
These attributes lead to several problems...
"Large, complex projects" suffer from two particular problems1. Lack of alignmentIt is much more difficult to create and maintain aligned intent, which means it's more difficult to coordinate efforts to achieve any overall target outcome.
This becomes worse with some multi-year programs as the sheer length of time tends to lead to a lack of continuity of leadership and management.

2. OverconfidenceNovices in "large, complex projects" tend to be overconfident in the predictability of humans, technology and context, that is, in the comprehensibility of both the problem and solution.

This overconfidence leads to a specific questionable belief: If it works on paper, then it must work in practice, that is, documents are a reliable form of control.


This belief leads to a critical failure pattern with  "large, complex projects": Waiting until late in the life cycle to test the integration of systems, humans, and processes.


I see the overconfidence as partly coming from an assumption that they're dealing with a technical problem rather than a socio-technical problem (multiple teams, vendors, sites, business processes, etc.).

Another way of describing this is considering the difference between a "bicycle system" and a "frog system".  With a bicycle, if we break everything down into pieces, it can be assembled into a working result in a predictable manner.  We can't do the same with a frog.  "Large, complex" projects in the enterprise, with the multiple teams, vendors, sites, non-IT components, etc. are more like frogs than bicycles.


Next: Solutions for "large, complex projects" in the enterprise that don't work

Frog picture from https://flic.kr/p/coAoWb
Categories: Blogs

Understanding The Relationship Between RabbitMQ Exchanges, Queues and Bindings

Derick Bailey - new ThoughtStream - Fri, 11/14/2014 - 22:01

I haven’t spent that much time looking at how RabbitMQ really works, recently. Sure, I’ve been using it a lot but I was using rabbit.js on top of it, which hides a lot of the details on how it works. I was happy with rabbit.js for this reason – but I recently switched one of my apps over to LeanKit’s “wascally” (the worst name, EVAR!!!! :D). In the process of doing that, I realized that I still had no clue how RabbitMQ was really working with regard to queues, exchanges and binding them. 

So I dug in to the wascally specs, pulled out my RabbitMQ In Action book again, and started cramming information in to my feeble brain, hoping something would click. And just now, sitting in an airport on my way to a conference, something did click. Suddenly, the relationship between exchanges, queues and binding them with routing keys, makes sense. That’s not to say I perfectly understand the right combinations of things, yet… but that I at least have the right base line to work from, which will make the rest of it relatively easy to figure out. 

Rabbitmq enlightenment

The spark that revealed the relationships to me, was a slow but sure realization that I was creating and binding my exchanges and queues in the wrong place. It started as a confused, “what?”  and “why would I … ???” which ultimately turned in to me realizing I was doing things backward.

RabbitMQ Exchanges and Queues

Exchanges are the place to which you publish a message. They are responsible for routing the message to the appropriate queue, through the use of an exchange -> queue binding.

Think of an exchange like the postal service. You specify an address on your envelope when you want to mail something, but you do not drive to that person’s home and insert the envelope in to their mailbox, yourself. Rather, you then take your envelope to your local postal service center – the “exchange”. Once the envelope is in the hands of the postal service, you don’t worry about it anymore. You expect the postal service to figure out how to get your envelope to the destination. 

RabbitMQ works the same way. You “address” the “envelope” of the message with information about the exchange to which you are publishing, and with an optional routing slip (both the exchange and routing slip can be considered part of the “address”) and a few bit of other information. Once you send it to the exchange, you expect the exchange will route the message to the correct queue – the “inbox” or “mailbox”. Code on the other side is subscribed to the queue in question, and receives the message. 

When you publish a message to RabbitMQ, you are doing through an exchange. Even if you think you are publishing directly to a queue, you are actually publishing through the “default” exchange, which is bound directly to the queue that you specified.

Exchange -> Queue Bindings

The “routing” that happens inside of an exchange is done through bindings. Having an exchange and a queue in place is simply not enough. If you don’t have them bound together appropriately, your message will never go anywhere. You can drop your envelope in an out-bound mailbox, but if the postal service never comes to collect the outbound mail, it will never get to the intended destination.

When working with RabbitMQ, then, you need to have both an exchange and a queue, and these need to be bound together with routing information. The routing information used depends on the type of exchanges, routing keys and other things. I won’t go in to detail here, because I’m still figuring these things out. But in general, the binding is there to ensure the messages move from the exchange to the correct queue. 

Where Are They Defined And Bound??

In my experience, this default exchange has been one of the causes of my confusion about exchanges. I started my RabbitMQ experience by publishing directly to a queue – or so i thought – because that was how I did things with WebsphereMQ, 6 years ago. I thought I was publishing directly to a queue in RabbitMQ, and I got confused. I didn’t understand the point of exchanges, because I was always defining my exchanges and my destination queue within the publisher – exactly the wrong place to do it!

Having declared both my exchange and queue, and bound them together, within the publisher is a bit like me driving to my post office, handing them an envelope, then getting in a postal delivery truck and driving to the address on my envelope so that I can deliver it. It completely bypasses all of the benefit of having exchanges separated from queues.

It turns out, the exchange should be declared within the publisher, but the destination queue and binding should not. It is a much better idea to have the subscriber bind the queue to the exchange. By doing this, the publisher does not care at all about the destination. It only cares about the exchange to which it publishes. I only care about getting to my local post office. I don’t care how they get my envelope to the mailbox of my intended recipient.

On the subscriber side, things are a bit different and there is some debate here. I generally think the easiest way to set up the subscriber is to have it declare the exchange, the queue and the binding all together.

I think it’s possible to have the subscriber only declare the queue and exchange binding, but I think it is safer to have it declare the exchange as well. I’m not sure what would happen if the client declared a binding to an exchange that did not exist. It probably wouldn’t do anything, or would throw an error.

The debate, then, is whether or not the subscriber should declare the exchange and binding. It can be argued that the binding between exchange and queue is a completely separate concern from both the publisher and subscriber. I agree with this in principle, but this introduces other difficulties in RabbitMQ. Since there is no need for any kind of configuration or administration tool within RabbitMQ, and since the RabbitMQ protocol itself defines the ability to create exchanges, queues and bindings, I find the introduction of a 3rd tool to manage the bindings to be inconvenient at this point. I prefer to let my subscriber code manage this binding relationship. That does not mean I won’t change my mind in the future, but right now, this is my preferences.

Forward Progress, Finally

I’ve felt like I was missing a huge piece of the RabbitMQ puzzle, for the last year. I was declaring my exchange and queue inside of the publisher, wondering why I should even bother with exchanges – what was the point of them, if my publisher had to declare the exchange, queue and binding, after all? 

With this one realization – that only the exchange should be declared within my publisher – I seem to have turned around, mid-stream, and started to swim with the flow of RabbitMQ, instead of trying to work against it. I am hopeful that this revelation will help to further enhance my productivity with RabbitMQ, and increase the rate at which I learn more about the details and configuration of RabbitMQ, and ultimately the system design around RabbitMQ.

     Related Stories 
Categories: Blogs

The Three People Who Need to ‘Just Say No’

Agile Management Blog - VersionOne - Fri, 11/14/2014 - 17:07

Guest post by Scott Dunn, Rocket Nine Solutions

“We have too many things going on!” I hear this often from developers, product owners, even management. The problem is that we might be enabling this problem unknowingly (that does not, by the way, go away by itself). Depending on your role, I have three ideas for three different people: (1) the team, (2) managers and the PMO, and (3) the ScrumMaster.

At the team level, this complaint of too m­any projects or initiatives is very common. When I was a manager, my people would come up to me and say, “Jim in Marketing thinks I should be working on his request; Leonard just told me his changes were needed yesterday; and yet, we’re not even halfway done with Scotty’s list. Can you please tell me what’s the priority??”

Now that we’re doing Scrum, the priority is clear, right? There’s only one stack-ranked list from which to work. That gives us clarity, focus, limited work in progress, short feedback loops with everyone, and an increment of done-done potentially shippable software. Hallelujah!

Only, this is often not happening. No single list for a person, no limit on the work, no singing Hallelujah. The majority of companies and teams I’ve worked with miss out on this incredible win because they break a fundamental principle of Scrum… The team members are shared, or maxtrixed, with other teams or projects.

When stakeholders or management come to the product owner with a new request, the response from the product owner should be, “Let’s talk about where it fits in the backlog. Thanks; I’ll see where goes,” or perhaps, “Please tell me where it goes.”

Instead, these requests sometimes go to management or the PMO, and the manager or program manager says, “Okay, I’ll grab some people and get right on it.” But they’re grabbing people who are on dedicated Scrum teams. “Oh, that’s right – we’re doing Scrum. Okay, I’ll grab some people and form a new Scrum team and get right on it.”

And we’re back to people asking what’s priority, because “Jim thinks his team needs me about 75%, Leonard just told me their team needs me 100%, but my current team isn’t even halfway done with Scotty’s list. Can you please tell me what’s the priority??”

This matrixing is costing us not only in frustration at the team level. It’s costing us real dollars. According to a recent post on Inc.com, research shows the cost estimated at $450 billion dollars a year globally. How? A 40% drop in productivity, 50% longer to complete a single task, and up to 50% more errors. Doesn’t sound like something we want to do, right?

Too many things going on.

So what can we do to improve right now where we’re at? There are a couple of immediate actions, and perhaps a bigger, and more valuable, solution long-term.

First, product owners…

Tell the requestors “No.” In Henrik Kniberg’s wonderful overview video of the product owner role, he says practicing saying “No” goes a long way. I typically respond with a different version – “Yes, AND…” That is, “Yes, AND if you show me which of your previous requests this new request is more important than, we can do it at that point.” With tougher stakeholders, I’ll admit that I have said, “I’ll put it in the backlog right away!” I would do that (I don’t run around lying to executives), but I would sit on the request and wait if this the request was “urgent but not important.” Or perhaps they would soon be distracted by some new, bright and shiny thing. Ideal and healthy? Perhaps not. But there are some places that I felt I had to pick my battles and use my time responsibly and effectively for more value elsewhere in the company. And dealing with pushy or whiny Ryan from Department B wasn’t always it.

Managers and the PMO

Second action is for managers and the PMO – tell the requestors “No” by not starting up yet another team. The truth is, if you have 100 developers and testers in IT, then you should have about 10-14 teams. The numbers tell us that matrixing is costing us a massive lack of focus. It also costs you in lost opportunity. Rather than simply delivering the most valuable thing in a month, we often deliver the 1st – 12th most valuable things at the end of a year, with not much business value or feedback in the middle of that year. And value degrades. A customer-requested feature delivered long past peak demand is just not as valuable as delivered at the height of market/customer demand.

ScrumMasters

Thirdly, ScrumMasters and internal agile coaches should tell the requestors, “No,” by talking with stakeholders and decision makers about these costs. Over and over again, I hear those guiding the agile adoptions at their companies tell me they’re in a matrixed organization. They admit that they know it’s bad, but they’re hoping it will change. If we could be honest for a moment, if you’re in this situation, how long has it been this way? How long have you been hoping for change? As a change agent, have you shared this information with them? Is there any buy-in, a timeframe, or roadmap for change in this area? Most commonly, real organizational change (such as how work is fed to teams from the program and/or portfolio) begins with understanding and support from top executives and stakeholders.

One example of a framework for alignment at these levels is the Scaled Agile Framework® (SAFe™), which provides some nice guidance for addressing this problem. First, we work in terms of value streams, which certainly something the business should relate to, and all contributors within that value stream are on teams. A level above the team’s Product Backlogs is a Program Backlog with a product manager who is the single point of accountability for this larger span of work. Although this can be handled in pure Scrum with a chief product owner, figuring out how to implement this can be quite difficult. The following can take years, if ever, to figure out on your own (especially without external coaching, which most aren’t fortunate enough to have:

  • The “what” of a portfolio, strategic themes, vision and roadmap
  • The “how” of actual coordination and ynchronizing the work of many teams
  • Governance such as architecture or UX

SAFe gives you a running start in these critical, but typically shelved, areas.

Even more helpful to help executives understand and support agile is that SAFe gives a simplified and easy-to-understand (ok, easier) context and plan for your stakeholders and other key roles from whom you need support for change. Keep in mind that many feel that we are in the Late Adopter (from the Technology Innovation Adoption Curve) state of agile. These Late Adopters are typically risk-averse, don’t like change, and would like a plan – at least to start with. To tell them, “We’ll self-organize around how to do agile here and we’ll figure it out,” is probably not something they want to hear.

I hope you found some ideas in this article that you can take action on and help others in the organization do the same. If so, we can all get back to the good stuff – meaningful, challenging work that delights stakeholders and customers.

Categories: Companies

What is the Prerequisite of Success When Scaling Agility?

Agile Management Blog - VersionOne - Fri, 11/14/2014 - 17:02

Guest post by Timofey Yevgrashyn, Ciklum

Today more and more enterprises are looking into agile approaches as the answer to their needs. It doesn’t matter if a big company starts scaling agile with or without acting Scrum teams. What matters is the way the company goes from the “let’s-scale-agile” decision to the visible results of scaled agility.

Why scale agility? A few thoughts.

One could say that the apparent reason to apply scaled agile practices is when you have many agile teams that are cross-dependent and working on the same product(s). But it wouldn’t be true as in our practice; the number of people doesn’t drive managers to even start thinking about applying methods above the team level.

What interesting is, the decisions to scale agile often come from the top. The feeling of “something is wrong with the business” on the top level is much higher than that feeling at the team level where the use of Scrum or other agile methods usually starts.

While talking to more senior management, the problems could emerge through the main “pain” areas, which have been reported in a number of agile development studies. Briefly, these areas are quicker time-to-market, better quality of the product, higher engagement of people and higher productivity. One or another, or even combined pain feeling in these areas, leads the enterprise management start thinking about better methods or development practices that could help coordinate many agile teams. The company management itself, or with the help of consultants, realizes the need for scaled agile. And so they start looking for already known frameworks such as the Scaled Agile Framework® (SAFe™) or experiment with their way.

Scaling agility doesn’t come easy.

It is worth mentioning that an organization structure is the result of that company’s history. Rarely (not to say never) is organizational structure architected in a scalable way from the beginning.

To be able to deliver better software and respond to change at the same time, an enterprise should overcome the major pitfall in scaling an organization. This pitfall is the traditional mindset of project-oriented companies instead of value-oriented companies.

Many of you have seen the triangles picture. It shows how the sequential phased approach (aka waterfall development) is compared to agile development methods in their view on three cornerstones: (1) scope, (2) time and (3) teams – resources and/or budget:

triangle image_Tim

Many organizations with a desire to scale agility still approach projects in the old, traditional way. The business part defines the total scope, and the software development part (or a vendor) is expected to estimate and plan how many resources and months they need to deliver what they were told.

At the same time, the agility of an organization means delivering what users need and cutting out waste from the original ideas to maximize value. This demands that an organization build a structure that, at regular pace, delivers a working product and then just balance out the decisions of what to do next in order for the business to develop.

What companies do before scaling.

Organizing people and communications around value streams is one of the major changes that happens when a company decides to scale agility. Even if the decision “let’s scale agility” is supported from the very top level, and everyone is eager to achieve this state quickly, there’s still a long way to go.

In our practice, this change requires management and key people to review how the organization earns money and delivers value. Often the discussion on how to organize teams around value streams requires changes in the team’s composition. This leads to the human factor and necessity to take this change carefully.

While the top level, or portfolio level, usually focused on value streams alignment, the other part of the organization should review roles, responsibilities and authority levels. Looking on the team and program levels, the need for coordination between teams emerges and, thus, different roles are needed. Nevertheless, these new roles should not be mapped directly to the old company structure. Built from the agility point of view, the duties list for these roles would include: serving, coordinating, and enabling as the primary drivers.

SAFe produces recommendations on what functions are needed on different levels. But like any other framework (LeSS, DAD or simple Scrum), SAFe should be used with care and deep understanding of why the role is needed and what value it brings to the enterprise.

Above organizing around value streams, it’s worth the reminder that an actual software development team is just an intermediate phase from the end-to-end value stream point of view. This makes dependencies on the upstream (usually product analysis and requirements preparation activities), requiring a seamless integration with the downstream (usually stabilization, packaging or deployment).

While both boundaries are meaningful, alignment with the upstream – and especially with business decisions – is critical. As our head of consulting mentioned once to a client, “There is nothing worse than having a great software development team with a great and optimized agile processes, doing wrong things.”

Such initial misalignment in practices makes it apparent that the organization should learn additional methods for content/requirements handling and practices at the program and portfolio levels to support agility at scale. As experience shows, it takes time for those responsible for the content of the product to adapt these methods to a degree where agile teams see the difference between before and now.

Furthermore, if your organization hasn’t had a rhythm of delivering working and demo-able product on team levels, it is risky to launch a joint program with many teams working together and call it scaled agility. The gravity of non-delivering could pull back any well-educated and well-coached organization.

A Program cadence (sometimes called “a release”) is a higher level of the rhythm and alignment achieved when business, content and delivery people can agree on what, when and how. Even if an organization plans their first-ever release cycle together with starting sprints on the teams level, still it would take time before management in charge of the “let’s-scale-our-agility” idea could clearly see benefits and results.

Something would go wrong, definitely.

It seems quite a clear path of activities that an enterprise goes through from decision to actual scaled agility:

  • Reorganizing around value streams;
  • Identifying the missing or new roles and responsibilities;
  • Embracing additional Backlog handling methods on all levels (Portfolio, Program, Team);
  • Achieving a rhythm of delivering working and demo-able results and then aligning on the high-level cadence such as a release;
  • Adapting continuous culture change and keeping up the momentum
  • Our observations show that even having a clear guideline and proven scaled agile framework in-hand, many companies will deviate and take this path differently.

One of the major obstacles has always been, and remains, company politics and old-fashioned culture that oppose the initiative of agility at enterprise-scale. Even with support from top management having a long walk to go, scaling agile challenges persons and ambitions, and causes fears and uncertainty with every decision.

One of the major signs will be a continuous culture change that regularly happens. Such a culture shift could manifest itself in small improvements handled by a team daily. Or it could be regular, healthy and actionable retrospectives, or in a high-level alignment that allows many people to plan together and deliver what’s intended, without a big overhead.

The motto that helps my colleagues and myself be patient on this is “Scaled agility is not the goal, rather a direction.” Thus, even being armed with clear steps and frameworks, we continue helping our clients incrementally to gain value from transforming to scaled agility by inspect-and-adapt all the way.

 

Categories: Companies

The Next Generation of Project Management

Agile Management Blog - VersionOne - Fri, 11/14/2014 - 17:01

Guest post by Chuck Cobb, Breakthrough Solutions/The Business Excellence Group

The concept of agile project management is very rapidly evolving and will have a significant impact on the project management profession, causing us to rethink many things we have taken for granted about what “project management” is for a long time.  However, we are in the very early stages of that transformation and there is a lot of confusion that exists in today’s world between the agile and traditional plan-driven project management (aka “waterfall”) communities. Many people see them as competitive and mutually-exclusive alternatives.  Some of the factors that contribute to this confusion are:

  • Many stereotypes and misconceptions exist about both agile and traditional plan-driven project management
  • Agile and traditional plan-driven project management principles and practices have been treated as separate and independent domains of knowledge with very little or no integration between the two
  • Agile and “waterfall” have been treated as binary, mutually-exclusive alternatives and many people make the mistake of force-fitting a project to one of those extremes

It’s no wonder that project managers and others might be confused by all of this!

A large portion of closing the gap that exists between the agile and traditional plan-driven project management communities can be attributed to a mindset change that includes:

1. Getting past a number of stereotypes and misconceptions that exist about both agile and traditional project management

2. Broadening our thinking about what “project management” is

Here are a few examples of the most popular stereotypes and misconceptions that exist about project managers:

1. Project managers are very “Command-and-Control” oriented.”

There is probably some reality behind this stereotype. Project managers are noted for getting results and sometimes that means being assertive and somewhat directive to set goals and manage the performance of project teams; sometimes that’s necessary.  Many times that behavior is expected of a project manager by the businesses in which they operate.  For example, if a project team is underperforming, the project manager is the one often held responsible and is expected to take corrective action to get the project on track.

2. Project managers are rigid and inflexible.

This stereotype also has some reality behind it, too. For many years, project managers have been held accountable for managing the costs and schedules of projects; and we all know that in order to meet cost and schedule goals, you have to control the scope of the project. That, in turn, requires a disciplined approach to managing change where change become the exception rather than the norm.

3. Project managers only know how to manage by the “waterfall” methodology.

The emphasis on managing costs and schedules for a long time has required accurately defining the requirements upfront, which leads to extensive use of “waterfall-style” methodologies that are based on trying to define project requirements in detail upfront before the project starts.  The emphasis on cost and schedule management is a significant reason why that approach continues to be used today.

4. Project managers are just not adaptive and cannot adapt to an agile environment.

I don’t believe that stereotype is correct, but it does require a significant shift in thinking for a project manager to make that adaptation. A good project manager knows that you have to adapt the project management approach to fit the problem rather than force-fit every problem or project to a single approach.

It should be apparent from all of these stereotypes that many of these behaviors are a function of the environment in which project managers operate, and are influenced by the expectations that businesses have of project managers. Creating a broader image of what project management is will also require influencing the environment and expectations on project managers.

Here are a few examples of the most popular stereotypes and misconceptions that exist about agile:

1. Agile projects are totally unplanned.

There is actually a lot of planning going on — It’s just a different kind of “just-in-time” planning that is more spread out through the project rather than being done heavily or totally upfront.  However, doing upfront planning is definitely not inconsistent with an agile approach. As an example, I was asked to rescue an agile project where, two years into the project, the project had no end in sight, and senior management had lost confidence that it would ever produce results. When we re-planned the project, we discovered that the scope of the effort was just too large to be done by a single agile team in any reasonable amount of time. If more upfront planning had been done, this problem would have been discovered much earlier.

2. Agile projects do not use any process and are totally uncontrolled.

That stereotype is also not accurate. Most agile projects use Scrum, which is a very well-defined process.  It’s a different kind of process – it is much more empirical and adaptive rather than rigid and prescriptive, but it is a well-defined process. It is also not difficult to apply whatever level of control you want to an agile project.  As an example, a few years ago I managed a large U.S. federal government project for the Federal Aviation Administration. We were able to create a hybrid model with a sufficient level of control to meet government contracting requirements, but at the same time provide a sufficient level of flexibility to further define detailed requirements with an agile development process.

3. There is no project management required for an agile project.

Although there may be no one with the title of “project manager,” in an agile project at the team level, there is plenty of project management going on.  It’s just a different kind of project management and the project management functions are distributed over a number of different people on the team rather than being held by one particular individual called a “project manager.”

  • The product owner in a Scrum project performs many project management functions by setting the direction and priorities for the project and making decisions as the project progresses; he or she is responsible overall for the success or failure of the project from a business perspective.
  • The ScrumMaster performs some project management functions by facilitating the team and the process, and is also responsible for removing obstacles that impede the team’s performance.
  • Everyone on an agile team performs some very basic project management functions in planning, estimating, and managing their own work, as well as tracking and reporting progress. Each member of the team as a whole must also coordinate and integrate all the activities of the team.

4. Documented requirements are not consistent with agile. Documentation is still useful in an agile project where it adds value.  Of course, documentation should not be done for the sake of creating documentation; however, there are many situations where it makes sense to document requirements in an online agile project management tool using a simple user story format as the project progresses.

In “The Next Generation of Project Management,” we need to:

  • Broaden our thinking about what “project management” is and think of it as a function, not a discrete role associated with someone called a “project manager,”
  • Develop a fully integrated agile project management approach that blends both agile and traditional plan-driven approaches in the right proportions to fit a given situation when necessary, and
  • View agile and traditional plan-driven project management approaches as complementary to each other rather than competitive

This is a significant challenge and will require a lot of new thinking, but it can be done.

It feels very similar to an evolution that took place when I worked in the quality management profession in the early 1990s. Up until that time, the primary emphasis in quality management had been on ‘quality control’ and inspection; the image of a ‘quality manager’ was heavily based on managing those functions.

  • The predominant quality management approach at that time was based on inspecting products before they were released from production prior to shipping to the customer and rejecting any that didn’t meet quality standards. It’s easy to see how that approach was inefficient because it resulted in a lot of unnecessary rework to correct problems after the fact and it also wasn’t that effective because any inspection approach is based on sampling – and it’s impractical to do a 100% sample. For that reason, this approach can result in a very mediocre level of quality.
  • A far better approach was to go upstream in the process and eliminate defects at the source by designing the process to be inherently more reliable and free of defects. That didn’t mean that the prior emphasis on quality control and inspection was obsolete and eliminated; it was just not the only way to manage quality, and wasn’t the most effective approach in all situations.
  • At the same time, “quality” took on a much broader meaning. It was no longer sufficient to produce products that were free of defects; producing products that were free of defects became just ‘table stakes’ to play in the game.  To be successful, companies really needed to go beyond that and produce products that really delighted their customers.

That was a gut-wrenching change for many in the quality management profession. Instead of being in control of quality and being the gatekeeper with the inspection process, a good quality manager needed to become more of a coach and a consultant to influence others to build quality into the way they did their work.  This changed the nature of the work dramatically for many in the quality management profession and eliminated a number of traditional quality management roles that were based primarily on the old quality control-and-inspection approach.  The similarity to the changes going on in the project management profession should be apparent:

  • Project managers needed to recognize that simply controlling the costs and schedules of a project was no longer sufficient; the project had to be successful from the perspective of producing value for the customer as well.
  • To be successful in more uncertain environments and focus on producing value, project managers need to be able to take an adaptive approach that is appropriate to the level of uncertainty in the project.
  • Finally, they need to give up some of the control that has become associated with the project management profession – in some cases, they may need to become more of a coach and a consultant to influence others rather than being in absolute control of a project.

This can dramatically change the role of a project manager and, in some situations, the role of a project manager as we’ve known it may no longer exist.  For example, at a team level in an agile project, you probably won’t find anyone with a title of “project manager” because the project management functions have been absorbed into other roles and are done very differently.  That doesn’t mean that project management is no longer important, but it may cause us to dramatically rethink what “project management” is in a much broader context than the way we may have thought about it in the past.[1]

About Chuck

Chuck Cobb is an Adjunct Professor at Boston University where he will be teaching a brand new, graduate-level course on agile project management. He is passionate about helping close the gap between the agile and traditional project management communities. To that end,

  • He has published two books on agile project management (a third book entitled “The Project Manager’s Guide to Mastering Agile” will be published in early 2015 by Wiley Publishing)
  • He has written over 50 articles on his blog site and has recently developed an online training course to help project managers learn how to develop an adaptive approach to project management that blends agile and traditional project management principles and practices in the right proportions to fit any situation

Chuck has worked with VersionOne for a number of years and has devoted a portion of his new book to using the VersionOne agile project management tool as an example of the importance of using tools in an enterprise-level agile project management strategy.

[1] Cobb, Charles, “The Project Manager’s Guide to Mastering Agile – Principles and Practices for an Adaptive Approach, Wiley Publishing, 2015

Categories: Companies

Excuse me – are you sprint ready?

Agile Learning Labs - Fri, 11/14/2014 - 16:12

A common complaint I hear from scrum teams is: we didn’t finish all the stories we committed to deliver in the sprint. While there are many reasons for this – one often-overlooked one is: The user stories were not ready to enter the sprint in the first place. The solution is for the scrum team to decide which stories are sprint ready before the sprint planning meeting even starts.

The sprint ready vote happens during story time (aka the product backlog grooming meeting). Sprint ready means that the team is confident that they can accomplish the story in one sprint. They have:

  • Confirmed and agreed on the acceptance criteria
  • Estimated the size
  • Confirmed all the story’s dependencies are complete
  • The story is small enough to be comfortably completed in a sprint (with all surrounding required processes)

I know this sounds amazing, too good to be true, even. Think about how the chances of work being completed in one sprint would increase if all those aspects of story preparation were completed before the team started? I know – mind-blowing!

So how do you get there? A few small changes in story time and sprint planning make this possible.

  1. When the team is having g story time, discuss whether the acceptance criteria is clear. If not, then the story is not sprint ready.
  2. When the team estimates the size of the story, make sure they are taking into account all work required to make a story production ready.
  3. Is the size estimate small enough that the team could complete four to six stories of that size in a week? If not, the story needs to be split into smaller stories and it’s not sprint ready.
  4. Review any dependencies on the development work. If there are outstanding dependencies– then it’s not sprint ready.

What if a story isn’t sprint ready? The development team and the product owner can discuss what’s needed to get it sprint ready, and then take the appropriate actions. This means that story might be presented in story time more than once, which is fine – remember refinement is part of the process.

When it comes to sprint planning, the product owner can only offer sprint ready stories to the team. They should offer them one at a time until the team says “Enough!” The product owner should not walk into sprint planning with the sprint already loaded with a whole set of stories, as this creates an unhealthy pressure on the team.

That’s it! It’s really pretty simple – however as with most things scrum – not always easy. So try it out. You don’t have much to lose, except maybe those undone stories.

Cheers,

Cathy Simpson

Categories: Companies

The Agile Reader – Weekend Edition: 11/14/2014

Scrumology.com - Kane Mar - Fri, 11/14/2014 - 07:38

You can get the Weekend Edition delivered directly to you via email by signing up http://eepurl.com/0ifWn.

The Weekend Edition is a list of some interesting links found on the web to catch up with over the weekend. It is generated automatically, so I can’t vouch for any particular link but I’ve found the results are generally interesting and useful.

  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • Agile is Not Only for Software Projects #agile #scrum #pmot #lean
  • RT @yochum: Agilitrix – Michael Sahota: People over Process #agile #scrum
  • What kind of rewards should we use in our organisations? – #performancereviews #agile #scrum
  • ☆★☆ JOB ALERT ☆★☆ #Job #Dallas – IT Business Analyst/PM – Agile/Scrum view full details
  • RT @scrumology: Do you know how to scale Scrum? #Scrum #Agile
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • You need @trello! It’s like organization powered by rocket fuel. (And it’s free!) #scrum #agile #projectmanagement
  • #scrum #DesingThinking #agile Caceres 4 de diciembre en el @CEEI_Ex
  • RT @antorecio: #scrum #DesingThinking #agile Caceres 4 de diciembre en el @CEEI_Ex
  • The four key tenets of #Scrum, still not followed enough #agile
  • RT @dozylocal: A lot (if not all) of this resonates with me. Nice summary :-) #agile #lean #scrum #mvp
  • Agile Scrum Master – RBC – Toronto, ON #JOBS #IT
  • RT @antorecio: #scrum #DesingThinking #agile Caceres 4 de diciembre en el @CEEI_Ex
  • Agile Canada » Agile Scrum Master – RBC – Toronto, ON: PMI certification as an Agile pra… #Canada #Agile #Jobs
  • RT @fancydev: A Scrum Master’s Practical Toolbox for creative Scrum Masters. @Scrumdotorg #agile #scrum http://t.co…
  • #LargeScaleScrum is now published on a website by Craig Larman and @basvodde #scrum #agile
  • OMG I was just tweeting :o I miss you twitter, will see you soon :) And agile/scrum rant is coming after #provoked
  • RT @yochum: Agilitrix – Michael Sahota: People over Process #agile #scrum
  • Denver CSM Class 12/8-9 w/ @skipangel #Scrum #Agile @RallySoftware http://t.co/2HqYgV1wXv
  • RT @yochum: On Software Development, Agile, Startups, and Social Networking – Isaac Sacolick: The Death of M… #agi…
  • RT @Astbrant: Great blog about UX… #RomanPichler #UXjayway #UX #Agile #Scrum
  • Agile Canada » Agile Scrum Master at RBC (Toronto, ON): agile team of 10 -15 cross functi… #Canada #Agile #Jobs
  • RT @JeanPierreTran: Waterfall vs. Agile vs. Lean, expliqué en une image #leanstartup #scrum
    http://t.co/7ahfhLu9yz
  • Subscribe now for my Q&A Agile Retrospectives Webinar: #scrum #agile #retrospectives
  • I liked a @YouTube playlist Agile Training (Scrum)
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • RT @scrumology: Do you know how to scale Scrum? #Scrum #Agile
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • Some recommendations for managing an outsourced software project. #pmot #agile #scrum
  • RT @lgoncalves1979: Subscribe now for my Q&A Agile Retrospectives Webinar: #scrum #agile #retrospectives
  • Throw Scrum Master out the window #agile #pmot #scrum
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • RT @vanesa_tejada: En RUMBO buscamos Scrum Masters, ¿te interesa? Entonces aquí tienes mas detalles: #empleo #agile …
  • Agile Scrum Master: RBC: “Purpose: Global Wealth Management Applications requires an… #software #jobs #toronto
  • Agile Canada » Project Manager/Agile Scrum Manager at More -> (Halifax, NS): management p… #Canada #Agile #Jobs
  • RT @MichikoDiby: Throw Scrum Master out the window #agile #pmot #scrum
  • Scrum Expert: Make Impacts Not Software #agile #scrum
  • RT @yochum: Scrum Expert: Make Impacts Not Software #agile #scrum
  • ☆★☆ JOB ALERT ☆★☆ #Job #Dallas – IT Business Analyst/PM – Agile/Scrum view full details
  • Agile by McKnight, Scrum by Day is out! Stories via @agilelynn @TheAgileNetwork @AgileUniversity
  • Emergent Design, The best results emerge from self-organizing teams… #agile #scrum @ScrumAlliance @martinalaimo
  • RT @MichikoDiby: Throw Scrum Master out the window #agile #pmot #scrum
  • 15% off for our Agile and Scrum workshop in Dublin, Ireland on 5th Dec 2014 at Dublin 8, Ireland #Dublin 8
  • Kanban & Agile Scrum: Is the Basic Concept Similar? #pmot #p http://t.co/HyCbcIEX57
  • RT @TalD: The Agile/ Scrum/ Lean Daily is out! Stories via @ADTmag @bradwilson @MariaMatarelli
  • RT @amckni: Agile by McKnight, Scrum by Day is out! Stories via @agilelynn @TheAgileNetwork @AgileUniversity
  • RT @lgoncalves1979: Subscribe now for my Q&A Agile Retrospectives Webinar: #scrum #agile #retrospectives
  • Categories: Blogs

    Guest blog: Retrospectives with Wordles

    Ben Linders - Fri, 11/14/2014 - 00:32
    There is a danger with retrospectives that teams will end up just going through the motions and not use the valuable material to identify whether the actions agreed at the end of the retrospective are actually making an impact on the team’s capability to improve. Furthermore, the chances are that in a multi-team environment, there are common themes raised that if highlighted early, can aid a new team when starting up – learn from others misfortune! Following the Agile philosophy of transparency, here at the UK Ministry of Justice we have been using Wordles to really get the key messages across in a clear manner. Continue reading →
    Categories: Blogs

    How To Abandon Practices

    NetObjectives - Thu, 11/13/2014 - 22:19
    We often hear no one size fits all, but then no one explains how to abandon the practices of what they preach. While Scrum is a great team framework, not all of the Scrum practices mandates in the framework should be used all of the time. In an earlier blog, The Right Way to Do Scrum-But I talked about the need to continue achieving the purpose of the abandoned practice. In this blog I'll give...

    [[ This is a content summary only. Visit my website for full links, other content, and more! ]]
    Categories: Companies

    Agile is Not Only for Software Projects

    TV Agile - Thu, 11/13/2014 - 22:01
    This presentation demonstrates specific practices for applying Agile concepts to all sorts of projects. It starts by reviewing the Agile concepts and then provide examples of Agile practices (and tools) applied to both software and non software projects. It will also describe how this particular set of Agile practices (called Commitment-based Project Management – CBPM) […]
    Categories: Blogs

    Make Impacts Not Software

    Scrum Expert - Thu, 11/13/2014 - 21:56
    Software is everywhere today, and countless software products and projects die a slow death without ever making any impact. Today’s planning and roadmap techniques expect the world to stand still while we deliver, and set products and projects up for failure from the very start. Even when they have good strategic plans, many organizations fail to communicate them and align everyone involved in delivery. The result is a tremendous amount of time and money wasted due to wrong assumptions, lack of focus, poor communication of objectives, lack of understanding and misalignment ...
    Categories: Communities

    Gatineau Ottawa Agile Tour 2014 (GOAT#14)

    Notes from a Tool User - Mark Levison - Thu, 11/13/2014 - 19:42
    GATINEAU OTTAWA AGILE TOURAgile Pain Relief is proud to be the Platinum sponsor for Gatineau Ottawa Agile Tour 2014.

    On November 24th, an expected 200+ Scrum and Agile professionals will gather in Gatineau for a one day conference that will feature two keynote speakers, 16 professional presentations, and coaches clinic among the opportunities.

    The conference’s keynote speakers this year will be Christopher Avery and Richard Cheng. Christopher is a sought-after speaker, author, and business advisor on responsible leadership, teamwork, and agile change for companies like GAP, Wells Fargo, and eBay, known for his cutting-edge work to demystify and then develop practical team leadership skills. And Richard is a Certified Scrum Trainer (CST) and Project Management Professional (PMP) with deep expertise in bringing Agile to commercial, non-profit, and Federal clients.

    Mark Levison will be one of 16 additional high-quality featured presentations, where he will present his High Performance Teams Game.

    As a Coach and a Team Member, I sometimes come across teams that don’t seem to be able to achieve high performance, and the reasons why aren’t apparent. This can be frustrating for everyone when this happens, so I wanted to explore this deeper and understand the science of what happens inside Agile/Scrum Teams.

    From his research and study, Mark has developed a game that will help attendees learn from their own experience some of what it takes to build high performance teams.

    Full details are available on the http://goagiletour.ca website. Tickets are limited and selling fast, so don’t delay. $70 registration ($25 student) includes presentations, breakfast and lunch, and up to 5 PDUs.

     

    Categories: Blogs

    Mobile authentication with Xamarin.Auth and refresh tokens

    Jimmy Bogard - Thu, 11/13/2014 - 15:46

    An internal app I’ve been working with for a while needed to use OAuth2 (specifically, OpenID Connect) to perform authentication against our Google Apps for Your Domain (GAFYD) accounts. Standard OAuth 1.0/2.0 flows are made easy with the Xamarin.Auth component. Since OpenID Connect is built on top of OAuth 2.0, the Xamarin.Auth component could suffice.

    A basic flow for using OAuth with Google APIs would look like this:

    image

    But for our purposes, we have a mobile application that connects to our APIs, but we simply want to piggyback on top of Google for authentication. So our flow looks more like:

    image

    This all works great straight out of the box, very nicely. One problem however – the token returned by the Google servers is only valid for a short period of time – 30 minutes or so. You *could* ignore this problem in the API we built, and not validate that part of the JWT. However, we don’t want to do that. Because we’re now going over the interwebs with our API calls, and potentially over insecure networks like coffee shop wi-fi, we want a solid verification of the JWT token:

    • The token’s hash matches
    • The issuer is valid (Google)
    • The allowed audience is correct – we only accept client IDs from our app
    • The certificate is signed against Google’s public OAuth certificates
    • The token has not expired

    This becomes a bit of a problem – the token expires very soon, and it’s annoying to log in every time you use the app. The Xamarin.Auth component supports storing the token on the device, so that you can authenticate easily across app restarts. However, out-of-the-box, Xamarin.Auth doesn’t support the concept of refresh tokens:

    image

    Since the refresh token is stored on the device, we just need to ask Google for another refresh token once the current token has expired. To get Xamarin.Auth to request a refresh token, we need to do a couple of things: first, override the GetInitialUrlAsync method to request a refresh token as part of getting an auth token:

    public override Task<Uri> GetInitialUrlAsync ()
    {
    	string uriString = string.Format (
    		"{0}?client_id={1}&redirect_uri={2}&response_type={3}&scope={4}&state={5}&hd=foo.com&access_type=offline&approval_prompt=force",
    		this.AuthorizeUrl.AbsoluteUri,
    		Uri.EscapeDataString (this.ClientId),
    		Uri.EscapeDataString (this.RedirectUrl.AbsoluteUri),
    		this.AccessTokenUrl == null ? "token" : "code",
    		Uri.EscapeDataString (this.Scope),
    		Uri.EscapeDataString (this.RequestState)
    	);
    
    	var url = new Uri (uriString);
    	return Task.FromResult (url);
    }
    

    The format of the URL is from Google’s documentation, plus looking at the behavior of the existing Xamarin.Auth component. Next, we create a method to request our refresh token if we need one:

    public virtual Task<int> RequestRefreshTokenAsync(string refreshToken)
    {
        var queryValues = new Dictionary<string, string>
        {
            {"refresh_token", refreshToken},
            {"client_id", this.ClientId},
            {"grant_type", "refresh_token"}
        };
    
    	if (!string.IsNullOrEmpty(this.ClientSecret))
    	{
    		queryValues["client_secret"] = this.ClientSecret;
    	}
    
    	return this.RequestAccessTokenAsync(queryValues).ContinueWith(result =>
    	{
    	    var accountProperties = result.Result;
    
    	    this.OnRetrievedAccountProperties(accountProperties);
    
            return int.Parse(accountProperties["expires_in"]);
    	});
    }
    

    I have a pull request open to include this method out-of-the-box, but until then, we’ll just need to code it ourselves. Finally, we just need to call our refresh token as need be before making an API call:

    var account = AccountStore.Create().FindAccountsForService("MyService").FirstOrDefault();
    
    if (account != null) {
        var token = account.Properties["refresh_token"];
        var expiresIn = await authenticator.RequestRefreshTokenAsync(token);
        UserPreferences["tokenExpiration"] = DateTime.Now.AddSeconds(expiresIn);
    }

    In practice, we’d likely wrap up this behavior around every call to our backend API, checking the expiration date of the token and refreshing as needed. In our app, we just a simple decorator pattern around an API gateway interface, so that refreshing our token was as seamless as possible to the end user.

    In your apps, the URL will likely be different in format, but the basic format is the same. With persistent refresh tokens, users of our mobile application log in only once, and the token refreshes as needed. Very easy with Xamarin and the Xamarin.Auth component!

    Some minor complaints with the component, however. First, it’s not a Xamarin.Forms component, so all the code around managing accounts and displaying the UI had to be in our platform-specific projects. Second, there’s no support for Windows Phone, even though there are issues and pull requests to fill in the behavior. Otherwise, it’s a great component that makes it simple to add robust authentication through your own OAuth provider or piggybacking on a 3rd party provider.

    Post Footer automatically generated by Add Post Footer Plugin for wordpress.

    Categories: Blogs

    How do you know that your product works? Slides from my LKCE14 keynote.

    Henrik Kniberg's blog - Wed, 11/12/2014 - 21:44

    Here are the slides for my keynote How do you know that your product works at Lean Kanban Central Europe, Hamburg.

    I travelled with Emma (6 yrs), she’s been wanting to travel with me (alone, without her 3 siblings…) for a long time, so she’s really happy! Thanks Mary & Tom Poppendieck for being her bonus grandparents during the whole trip :)

    Some sample slides & pics below.

    LKCE14

    LKCE14

    How are you improving people's lives

    What you measure is what you get

    Don't rely ONLY on hard data 

    How do you know that your product works

    Categories: Blogs

    Agile Through a Matrix Lens

    Tyner Blain - Scott Sehlhorst - Wed, 11/12/2014 - 21:39

     

    glasses lens

    “Agile” is something most teams do wrong*, without realizing they’re doing it wrong.  A good 2×2 matrix acts as a lens, helping to convert information into insight.  Let’s apply this lens to agile as applied within a company, and see if it helps people decide to do things differently.

    When You Say Agile, What Do You Mean?

    There may be as many definitions of agile as there are teams practicing agile development.  Generally, people are talking about iterating in what they do.  Instead of having a long, throw it over the wall process, out of which emerges a deliverable; a team will have a series of shorter iterations where they engage stakeholders and otherwise rethink what they are doing to course-correct and “get smarter” about what they are doing.  The Wikipedia page on agile is pretty comprehensive.

    Most teams think about agility in terms of how their development teams manage their process.  When “going agile” is the only thing you do, your product does not magically become more successful.  Some teams** think about what it means to be agile when determining what the development team should be doing in the first place. My epiphany was in realizing that these are two separate decisions an organization can make.

    A 2×2 Matrix of Agile

    When an organization can make two discrete decisions about being agile in how they create products, it results in four possible outcomes.  A 2×2 matrix can act as a powerful lens for exploring these decisions.  Our first step is to define our two axes.

    Requirements – how are they treated within the organization / by the team?

    • Requirements and expectations are immutable – this is the typical expectation within a large bureaucracy; someone built a business case, got funding, and allocated a team to deliver the product as-defined.
    • Requirements continually revisited – this is what we see nimble teams doing – at different levels of granularity, context, and relevance; at a low level, this is A|B testing and at a high level this is a pivot.

    Development process cadence – how frequently does the team deliver***?

    • Infrequent delivery – there is no one size fits all measure to define infrequent vs. frequent; some companies will have fast-moving competitors, customers with rapidly evolving expectations, and significant influence from evolving technology – others will not (for now).
    • Frequent delivery – the precise delineation from infrequent to frequent delivery is contextually dependent.

    With these two axes, we can draw a matrix.

    Agile 2x2 Matrix

    A subordinate message that I couldn’t resist putting into the matrix is that it is harder to do your job in an agile way.  I think you could pedantically argue that agile is easier – by saying it is easier to deliver the equivalent results when your process is agile.  And that’s true.  The point is to deliver a more successful product, which is harder than delivering a less successful product.  An agile approach makes that task easier.  Maybe another way to think about it – if your team is not capable of delivering a good product, going agile will make that more obvious, faster.

    Living in Boxes

    Everyone can map their team into one of the four boxes.  That’s the power of this sort of abstraction.

    Here’s where I can use your help: What are better names for these boxes?  I have satisficed with these names, but they could be better.  Please comment below with proposed alternatives, because I’ll be incorporating this lens into other aspects of my work, and I want it to be better than it currently is.

    Waterfall as Practiced

    barrel

    While there are some teams which consciously choose agile because of the planning benefits or perceived risks to quality, I believe that most waterfall teams are still waterfall either because they haven’t chosen to revisit their process choices, or they tried and failed.  Perhaps their instructors weren’t good, perhaps the team was not equipped to make the shift.  My guess is that their organizations were unwilling or unable to support any change in the bureaucratic status quo, effectively making it impossible for the teams to succeed.

    BUFD & BUFR (Buffed and Buffer)

    set in stone

    BUFR is an acronym for big up-front requirements, and BUFD is the equivalent for big up-front design.  Both of them are labels assigned as part of the decade-old war between interaction design and extreme programming.  Conceptually, the battle is between rationalists and empiricists.  In a nutshell, the requirements are Defined (capital Defined), then the team applies an agile development methodology (mostly) to incrementally build the product according to the requirements.

    This is another area where can explore more – what are requirements, what is design, who owns what? My main point is that the developers, while going through the agile motions – even when getting feedback – are only realizing some of the benefits of agile.  Yes, they can improve the effectiveness of their particular design or implementation at solving the intended problem.  Yes, they can avoid the death-march.

    The problem is that the requirements are set in stone metaphorically.

    At the end of the day, the team is empowered to rapidly iterate on, and change how they choose to solve the target (market) problems.  The team is not empowered to rapidly change their minds about which market problems to target.

    When agile is being introduced to a larger organization, as a grass-roots initiative starting with a development team, this is corner the team will find themselves in.

    Req’s Churn or Glacial Dev

    bingo ballsglacier

    I struggle for the right way to describe the situation where the people responsible for determining the requirements are getting market feedback and changing their requirements, which the people responsible for creating the product are unwilling or unable to accept changes from the initial plan.

    From the development team’s point of view, “the product manager can’t make up his mind – we are just churning, without getting anything done!”

    From the product manager’s point of view, “the development team is too slow, or intransigent, and can’t seem to keep up.”

    There’s only one environment where this approach is somewhat rational – outsourced development with limited trust.  When the relationship between the product management / design team, and the product creation / test team is defined by the contract, or the two teams do not trust each other, the only reasonable way to make things work is to establish explicit expectations up front, and then deliver to those specifications.  Note that the specifications typically include a change-management process, which facilitates reaching an agreement to change the plan.  The right way to make this type of relationship work is to change it, but if you’re stuck with it – this is your box.

    Agile as Intended

    Yin Yang

    Ah, the magic fourth box. Where rapid delivery leads to rapid learning which leads to rapid changes in the plan.  The success of agile is predicated on the assumption that as we get feedback from the market, we get smarter; as we get smarter, we make better choices about what to do next.

    This is what enables a sustainable competitive advantage, by enabling you to sustainably differentiate your product from competition, rapidly adapt to changing customer expectations and market conditions.  Effectively, you are empowered to change what you choose to do, as well as how you choose to do it.  This is what agile product management is about – enabling business agility.

    A winning strategy involves selecting an attractive market, developing a strategy for how you will compete within that market, then developing a product (or portfolio) roadmap which manifests the strategy, while embodying the vision of the company.  It is possible to do this in any corner of the matrix (except the upper left, in my opinion).  The less willing you are to rely on your ability to predict the future accurately, the more you will want to be in the upper right corner.

    Conclusion

    There isn’t a particularly strong argument against operating your team in the upper right hand corner of the box, Agile as Intended.  The best argument is really just “we aren’t there yet.”  From conversations I’ve had with many team leaders, they seemed to think that getting to the lower right corner was the right definition of “done.”  They thought they were “doing agile” and that there wasn’t anything left to change, organizationally.  And they wondered why their teams weren’t delivering on the promise of agile.  It’s because they weren’t there yet.

    Hopefully this visual will help drive the conversation forward for some of you out there.  Let me know if it helps light bulbs go off.

    Attributions and Clarifications

    Special thanks to Prabhakar Gopalan, who introduced me to The Power of the 2×2 Matrix, a really good book for framing ideas, in his craftsman product management training class.

    *Agile isn’t a noun really something you do, agile is an adverb describing how you do something.  English is a funny language, and “doing agile” is generally used to reflect developing a product in an agile manner.  Sometimes it is important to point this out – particularly when you’re trying to help people focus on the product and not the process (like here), but for this article, I didn’t want to dilute the other messages.  As a bonus, the people who would be tweaked by the use of agile as a noun are generally people who “get it” and I like the idea that they read the whole article, just to see this caveat.  Thanks for reading this :).

    **This is based on anecdata (thanks Prabhakar for the great word), but my impression is that small companies commonly do this – think Lean Start Up – and large companies rarely do this.  I suspect this is more about the challenge of managing expectations and otherwise navigating a bureaucracy built to reward execution against a predetermined plan.

    ***Definitions of “deliver” make for a great devil is in the details discussion too – do you deliver to end-customers or internal stakeholders?  What if your existing customers refuse to update every month?  How many versions of your product do you want in the field?  Another great topic – but not the focus of this article?

    Categories: Blogs

    Knowledge Sharing


    SpiraTeam is a agile application lifecycle management (ALM) system designed specifically for methodologies such as scrum, XP and Kanban.