Skip to content

Feed aggregator

R: ggplot – Show discrete scale even with no value

Mark Needham - Sat, 06/27/2015 - 00:48

As I mentioned in a previous blog post, I’ve been scraping data for the Wimbledon tennis tournament, and having got the data for the last ten years I wrote a query using dplyr to find out how players did each year over that period.

I ended up with the following functions to filter my data frame of all the mataches:

round_reached = function(player, main_matches) {
  furthest_match = main_matches %>% 
    filter(winner == player | loser == player) %>% 
    arrange(desc(round)) %>% 
    head(1)  
 
    return(ifelse(furthest_match$winner == player, "Winner", as.character(furthest_match$round)))
}
 
player_performance = function(name, matches) {
  player = data.frame()
  for(y in 2005:2014) {
    round = round_reached(name, filter(matches, year == y))
    if(length(round) == 1) {
      player = rbind(player, data.frame(year = y, round = round))      
    } else {
      player = rbind(player, data.frame(year = y, round = "Did not enter"))
    } 
  }
  return(player)
}

When we call that function we see the following output:

> player_performance("Andy Murray", main_matches)
   year          round
1  2005    Round of 32
2  2006    Round of 16
3  2007  Did not enter
4  2008 Quarter-Finals
5  2009    Semi-Finals
6  2010    Semi-Finals
7  2011    Semi-Finals
8  2012         Finals
9  2013         Winner
10 2014 Quarter-Finals

I wanted to create a chart showing Murray’s progress over the years with the round reached on the y axis and the year on the x axis. In order to do this I had to make sure the ’round’ column was being treated as a factor variable:

df = player_performance("Andy Murray", main_matches)
 
rounds = c("Did not enter", "Round of 128", "Round of 64", "Round of 32", "Round of 16", "Quarter-Finals", "Semi-Finals", "Finals", "Winner")
df$round = factor(df$round, levels =  rounds)
 
> df$round
 [1] Round of 32    Round of 16    Did not enter  Quarter-Finals Semi-Finals    Semi-Finals    Semi-Finals   
 [8] Finals         Winner         Quarter-Finals
Levels: Did not enter Round of 128 Round of 64 Round of 32 Round of 16 Quarter-Finals Semi-Finals Finals Winner

Now that we’ve got that we can plot his progress:

ggplot(aes(x = year, y = round, group=1), data = df) + 
    geom_point() + 
    geom_line() + 
    scale_x_continuous(breaks=df$year) + 
    scale_y_discrete(breaks = rounds)

2015 06 26 23 37 32

This is a good start but we’ve lost the rounds which don’t have a corresponding entry on the x axis. I’d like to keep them so it’s easier to compare the performance of different players.

It turns out that all we need to do is pass ‘drop = FALSE’ to scale_y_discrete and it will work exactly as we want:

ggplot(aes(x = year, y = round, group=1), data = df) + 
    geom_point() + 
    geom_line() + 
    scale_x_continuous(breaks=df$year) + 
    scale_y_discrete(breaks = rounds, drop = FALSE)

2015 06 26 23 41 01

Neat. Now let’s have a look at the performances of some of the other top players:

draw_chart = function(player, main_matches){
  df = player_performance(player, main_matches)
  df$round = factor(df$round, levels =  rounds)
 
  ggplot(aes(x = year, y = round, group=1), data = df) + 
    geom_point() + 
    geom_line() + 
    scale_x_continuous(breaks=df$year) + 
    scale_y_discrete(breaks = rounds, drop=FALSE) + 
    ggtitle(player) + 
    theme(axis.text.x=element_text(angle=90, hjust=1))
}
 
a = draw_chart("Andy Murray", main_matches)
b = draw_chart("Novak Djokovic", main_matches)
c = draw_chart("Rafael Nadal", main_matches)
d = draw_chart("Roger Federer", main_matches)
 
library(gridExtra)
grid.arrange(a,b,c,d, ncol=2)

2015 06 26 23 46 15

And that’s all for now!

Categories: Blogs

Creative Collaboration

Doc On Dev - Michael Norton - Fri, 06/26/2015 - 20:17
I had the pleasure of presenting at NDC Oslo last week and the additional privilege of co-presenting a collaboration workshop along with Denise Jacobs and Carl Smith.

Creative Collaboration: Tools for Teams from Doc Norton
In this workshop, we cover Fist to Five voting, 5x7 Prioritization, and Collaboration Contracts. We had around 30 attendees for the workshop, allowing us to create 4 groups of approximately 8 people each.

After some ice-breakers, groups came up with product ideas by mashing two random words together and using first to five voting to rapidly identify a product idea they could all agree on. This was easier for some groups than others. It was interesting to see the dynamics as some groups discussed each combination prior to voting, some groups created multiple options before voting, and other groups ripped through options and found their product in a manner of minutes (as intended). It is often difficult for us to give up old habits even in pursuit of a better way.

Next up was brainstorming and prioritizing a list of items that needed to be done in order to launch our new awesome concept at a key conference in only three months. We started with each individual member writing at least two items they thought were critically important to prepare for the conference. We then removed duplicate items for each group and used 5x7 prioritization to come up with the top most important items for each group. At the end of the process, teams agreed that the resultant priorities were good and many were surprised at how easy and equitable the process was.

Finally, each group took their top 4 items and ran collaboration contracts against them. We did this in two passes; running the basic contract and resolving conflicts. We had one group that ended up with no conflicts. The other groups worked through their conflicts in relatively short order and the quality of conversation was high throughout. One group realized that even after they resolved the obvious conflicts, they had one individual who was in a decision making role on all four items. While this is not technically a conflict on a contract, it does indicate an issue. After some additional discussion, they were able to adjust the overall contract to everyone's satisfaction and eliminate the potential bottleneck.

This was our first time delivering this workshop and I thought it went quite well.

I'm planning to add Parallel Thinking to the workshop along with a couple more games to create a solid half-day collaboration tools workshop that can work for teams or groups.

If you're interested in this workshop for your team, let me know. Maybe, if we're lucky, Denise and Carl can come along too.

Categories: Blogs

To be a Profession or to Unionize in the Software Industry?

Agile Complexification Inverter - Fri, 06/26/2015 - 18:49

Which form of industry growth would you prefer - why?

Which path leads toward the culture you desire in a software development organization?

This is a wonderful article on the topic - read it and discuss with your colleagues.


Programmers don’t need a union. We need a profession.  BY 
"Unions work best for commodity labor, and I use that term non-pejoratively. Commodity work is easily measurable and people can often be individually evaluated for performance. For example, a fishing boat operator is measured according to the quantity of fish she procures. A lot of very important work is commodity labor, so I don’t intend to disparage anyone by using that term. Commodity work can be unionized because there aren’t large and often intangible discrepancies in quality of output, and collective bargaining is often the best way to ensure that the workers are fairly compensated for the value they produce. Software is not commodity work, however. It’s difficult to measure quality, and the field is so specialized that engineers are not remotely interchangeable. When the work is difficult to measure and large disparities of quality exist, you have a situation in which a certain less-egalitarian (in the sense of allowing top performers to receive high compensation, because it’s essential to encourage people to improve themselves) and more self-regulatory structure is required: a profession.""A profession is an attempt to impose global structure over a category of specialized, cognitively intensive work where the quality of output has substantial ramifications, but is difficult (if not, in the short term, impossible) to measure, giving ethics and competence primary importance. A profession is needed when it’s clear that not everyone can perform the work well, especially without specialized training. Here are some traits that signify the existence of a profession."1) Ethical obligations that supersede managerial authority.
2) Weak power relationships.
3) Continuing improvement and self-direction as requirements.
4) Allowance for self-direction.
5) Except in egregious cases, an agreement between employee and firm to serve each others’ interests, even after employment ends. 



Categories: Blogs

What Creates Trust in Your Organization?

Johanna Rothman - Fri, 06/26/2015 - 17:16

I published my most recent newsletter, Creating Trustworthy Estimates, this past week. I also noted on Twitter that one person said his estimates created trust in his organization. (He was responding to a #noestimate post that I had retweeted.)

Sometimes, estimates do create trust. They provide a comfortable feeling to many people that you have an idea of what size this beast is. That’s why I offer solutions for a gross estimate in Predicting the Unpredictable. I have nothing against gross estimates.

I don’t like gross estimates (or even detailed estimates) as a way to evaluate projects in the project portfolio because estimates are guesses. Estimates are not a great way to understand and discuss the value of a project. They might be one piece of the valuation discussion, but if you use them as the only way to value a project, you are missing the value discussion you need to have. See Why Cost is the Wrong Question for Evaluating Projects in Your Project Portfolio.

I have not found that only estimates create trust. I have found that delivering the product  (or interim product) creates more trust.

Way back, when I was a software developer, I had a difficult machine vision project. Back then, we invented as we went. We had some in-house libraries, but we had to develop new solutions for each customer.

I had an estimate of 8 weeks for that project. I prototyped and tried a gazillion things. Finally, at 6 weeks, I had a working prototype. I showed it to my managers and other interested people. I finished the project and we shipped it.

Many years later, when I was a consultant, I encountered one of those managers. He said to me, “We held our breath for 6 weeks until you showed us a prototype. You had gone dark and we were worried. We had no idea if you would finish.”

By that time, I had managed people like me. I asked them for visual updates on their status each week or two. I had learned from my experiences.

I asked that manager why they held their breath. I always used an engineering notebook. I could have explained my status at any time to anyone who wanted it. He replied, “We so desperately wanted your estimate to be true. We were so afraid it wasn’t. We had no idea what to do. When you showed us a working prototype, that’s when we started to believe you could finish the project.”

They trusted my initial estimate. It’s a good thing they didn’t ask for updated estimates each week. I remember that project as a series of highs and lows.

That’s the problem with invention/innovation. You can keep track of your progress. You can determine ways to make progress. And, with the highs, your meet or beat your estimate. With the lows, you extend your estimate. I remember that at the beginning of week 5 I was sure I was not going to meet my date. Then, I discovered a way to make the project work. I remember my surprise that it was something “that easy.” It wasn’t easy. I had tracked my experiments in my notebook. There wasn’t much more I could do.

Since then, I asked my managers, “When do you want to know my project is in trouble? As soon as it I think I’m not going to meet my date; after I do some experiments; or the last possible moment?” I create trust when I ask that question because it shows I’m taking their concerns seriously.

After that project, here is what I did to create trust:

  1. Created a first draft estimate.
  2. Tracked my work so I could show visible progress and what didn’t work.
  3. Delivered often. That is why I like inch-pebbles. Yes, after that project, I often had one- or two-day deliverables.
  4. If I thought I wasn’t going to make it, use the questions above to decide when to say, “I’m in trouble.”
  5. Delivered a working product.

PredictingUnpredictable-smallEstimates can be useful. They can show you the risks. And, I’m sure that only having estimates is insufficient for building trust. If you want to learn more about estimation, see Predicting the Unpredictable: Pragmatic Approaches to Estimating Cost or Schedule.

Categories: Blogs

How Do I Know If Agile Is Working?

AvailAgility - Karl Scotland - Fri, 06/26/2015 - 17:10
Moving the queen

Moving the queen, Gabriel Saldana, CC BY-SA

“How do I know if Agile is working?” This is a question I’ve been asked a lot recently in one form or another. If not Agile, its Scrum, or Kanban or SAFe or something similar. My usual response is something along the lines of “How do you know of anything is working?” And there generally isn’t a quick and easy answer to that!

I’ve come to the view that Lean and Agile practices and techniques are simply tactics. They are tactics chosen as part of a strategy to be more Lean and Agile. And becoming more Lean and Agile are seen as important to make necessary breakthroughs in performance in order to deliver desired results.

With that perspective, then the answer to “How do I know if Agile is working?” is that you achieve the desired results. That’s probably a long time to wait to find out, however, as it is a trailing measure. It is necessary, therefore, to identify some intermediate improvements which might indicate the results are achievable, and leading measures can be captured to give hat earlier feedback.

The lack of a quick and easy answer to “How do you know if anything is working?” is often because Lean and Agile have been introduced as a purely tactical initiative, without any thought to how they relates to strategy, what measurable improvements they might bring, and how any strategy and improvements will lead to desirable results. In fact very few people (if any) know exactly what those desirable results are!

I’m increasingly trying to work the other way – what the Lean community call Strategy Deployment. For any transformation to work, everybody in the organisation needs to know what results are being strived for, what the strategic goals are that will enable the necessary changes to get there, and what measurable improvements will indicate progress. Then the whole organisation can be engaged in designing and implementing tactical changes which might lead to improvement. Everything becomes a hypothesis and an experiment, which can be tested, the results shared and adjustments made.

In other words, Strategy Deployment leads to organisations becoming laboratories, where Lean and Agile can inform hypothesis on strategies, improvements and tactics. I think its the secret sauce to any transformation, which is why I’ll be talking about it more at various conferences over the rest of the year.

The first one is Agile Cymru in a couple of weeks. There’s a few tickets left, and considering the line-up of speakers, and the ticket cost, its incredible value. I highly recommend going, and I hope to see you there!

 

Categories: Blogs

Git Subproject Compile-time Dependencies in Sbt

Xebia Blog - Fri, 06/26/2015 - 14:33

When creating a sbt project recently, I tried to include a project with no releases. This means including it using libraryDependencies in the build.sbt does not work. An option is to clone the project and publish it locally, but this is tedious manual work that needs to be repeated every time the cloned project changes.

Most examples explain how to add a direct compile time dependency on a git repository to sbt, but they just show how to add a single project repository as a dependency using an RootProject. After some searching I found the solution to add projects from a multi-project repository. Instead of RootProject the ProjectRef should be used. This allows for a second argument to specify the subproject in the reposityr.

This is my current project/Build.scala file:

import sbt.{Build, Project, ProjectRef, uri}

object GotoBuild extends Build {
  lazy val root = Project("root", sbt.file(".")).dependsOn(staminaCore, staminaJson, ...)

  lazy val staminaCore = ProjectRef(uri("git://github.com/scalapenos/stamina.git#master"), "stamina-core")
  lazy val staminaJson = ProjectRef(uri("git://github.com/scalapenos/stamina.git#master"), "stamina-json")
  ...
}

These subprojects are now a compile time dependency and sbt will pull in and maintain the repository in ~/.sbt/0.13/staging/[sha]/stamina. So no manual checkout with local publish is needed. This is very handy when depending on an internal independent project/module and without needing to create a new release for every change. (One side note is that my IntelliJ currently does not recognize that the library is on the class/source path of the main project, so it complains it cannot find symbols and therefore cannot do proper syntax checking and auto completing.)

Categories: Companies

Inspirational Quotes, Inspirational Life Quotes, and Great Leadership Quotes

J.D. Meier's Blog - Fri, 06/26/2015 - 05:51

I know several people looking for inspiration.

I believe the right words ignite or re-ignite us.

There is no better way to prime your mind for great things to come than filling your head and hear with the greatest inspirational quotes that the world has ever known.

Of course, the challenge is finding the best inspirational quotes to draw from.

Well, here you go …

3 Great Inspirational Quotes Collections at Your Fingertips

I revamped a few of my best inspirational quotes collections to really put the gems of insight at your fingertips:

  1. Inspirational Quotes – light a fire from the inside out, or find your North Star that pulls you forward
  2. Inspirational Life Quotes -
  3. Great Leadership Quotes – learn what great leadership really looks like and how it helps lifts others up

Each of these inspirational quotes collection is hand-crafted with deep words of wisdom, insight, and action.

You'll find inspirational quotes from Charles Dickens, Confucius, Dr. Seuss, George Bernard Shaw, Henry David Thoreau, Horace, Lao Tzu,  Lewis Carroll, Mahatma Gandhi, Oprah Winfrey, Oscar Wilde, Paulo Coelho, Ralph Waldo Emerson, Stephen King, Tony Robbins, and more.

You'll even find an inspirational quote from The Wizard of Oz (and it’s not “There’s no place like home.”)

Inspirational Quotes Jump Start

Here are a few of my favorites inspirational quotes to get you started:

“Courage doesn’t always roar. Sometimes courage is the quiet voice at the end of the day saying, ‘I will try again tomorrow.’”

Mary Anne Radmacher

“Do not follow where the path may lead. Go, instead, where there is no path and leave a trail.”

Ralph Waldo Emerson

“Don’t cry because it’s over, smile because it happened.”

Dr. Seuss

“It is not length of life, but depth of life.”

Ralph Waldo Emerson

“Life is not measured by the number of breaths you take, but by every moment that takes your breath away.”

Anonymous

“You live but once; you might as well be amusing.”

Coco Chanel

“It is never too late to be who you might have been.”

George Eliot

“Smile, breathe and go slowly.”

Thich Nhat Hanh

“What lies behind us and what lies before us are tiny matters compared to what lies within us.”

Ralph Waldo Emerson

These inspirational quotes are living breathing collections.  I periodically sweep them to reflect new additions, and I re-organize or re-style the quotes if I find a better way.

I invest a lot of time on quotes because I’ve learned the following simple truth:

Quotes change lives.

The right words, at the right time, can be just that little bit you need, to breakthrough or get unstuck, or find your mojo again.

Have you had your dose of inspiration today?

Categories: Blogs

R: Scraping Wimbledon draw data

Mark Needham - Fri, 06/26/2015 - 01:14

Given Wimbledon starts next week I wanted to find a data set to explore before it gets underway. Having searched around and failed to find one I had to resort to scraping the ATP World Tour’s event page which displays the matches in an easy to access format.

We’ll be using the Wimbledon 2013 draw since Andy Murray won that year! This is what the page looks like:

2015 06 25 23 47 16

Each match is in its own row of a table and each column has a class attribute which makes it really easy to scrape. We’ll be using R’s rvest again. I wrote the following script which grabs the player names, seedings and score of the match and stores everything in a data frame:

library(rvest)
library(dplyr)
library(stringr)
 
s = html_session("http://www.atpworldtour.com/en/scores/archive/wimbledon/540/2013/results")
rows = s %>% html_nodes("div#scoresResultsContent tr")
 
matches = data.frame()
for(row in rows) {  
  players = row %>% html_nodes("td.day-table-name a")
  seedings = row %>% html_nodes("td.day-table-seed")
  score = row %>% html_node("td.day-table-score a")
 
  if(!is.null(score)) {
    player1 = players[1] %>% html_text() %>% str_trim()
    seeding1 = ifelse(!is.na(seedings[1]), seedings[1] %>% html_node("span") %>% html_text() %>% str_trim(), NA)
 
    player2 = players[2] %>% html_text() %>% str_trim()
    seeding2 = ifelse(!is.na(seedings[2]), seedings[2] %>% html_node("span") %>% html_text() %>% str_trim(), NA)
 
    matches = rbind(data.frame(winner = player1, 
                               winner_seeding = seeding1, 
                               loser = player2, 
                               loser_seeding = seeding2,
                               score = score %>% html_text() %>% str_trim(),
                               round = round), matches)
 
  } else {
    round = row %>% html_node("th") %>% html_text()
  }
}

This is what the data frame looks like:

> matches %>% sample_n(10)
               winner winner_seeding                       loser loser_seeding            score                round
61      Wayne Odesnik            (4)                Thiago Alves          <NA>            61 64 1st Round Qualifying
4     Danai Udomchoke           <NA>            Marton Fucsovics          <NA>       61 57 1210 1st Round Qualifying
233    Jerzy Janowicz           (24)                Lukasz Kubot          <NA>         75 64 64       Quarter-Finals
90       Malek Jaziri           <NA>             Illya Marchenko           (9)        674 75 64 2nd Round Qualifying
222      David Ferrer            (4)         Alexandr Dolgopolov          (26) 676 762 26 61 62          Round of 32
54  Michal Przysiezny           (11)                 Dusan Lojda          <NA>         26 63 62 1st Round Qualifying
52           Go Soeda           (13)               Nikola Mektic          <NA>            62 60 1st Round Qualifying
42    Ruben Bemelmans           (23) Jonathan Dasnieres de Veigy          <NA>            63 64 1st Round Qualifying
31        Mirza Basic           <NA>              Tsung-Hua Yang          <NA>     674 33 (RET) 1st Round Qualifying
179     Jurgen Melzer           <NA>              Julian Reister           (Q)    36 762 765 62          Round of 64

It also contains qualifying matches which I’m not so interested in. Let’s strip those out:

main_matches = matches %>% filter(!grepl("Qualifying", round)) %>% mutate(year = 2013)

We’ll also put a column in for ‘year’ so that we can handle the draws for multiple years later on.

Next I wanted to clean up the data a bit. I’d like to be able to do some queries based on the seedings of the players but at the moment that column contains numeric brackets in values as well as some other values which indicate whether a player is a qualifier, lucky loser or wildcard entry.

I started by adding a column to store this extra information:

main_matches$winner_type = NA
main_matches$winner_type[main_matches$winner_seeding == "(WC)"] = "wildcard"
main_matches$winner_type[main_matches$winner_seeding == "(Q)"] = "qualifier"
main_matches$winner_type[main_matches$winner_seeding == "(LL)"] = "lucky loser"
 
main_matches$loser_type = NA
main_matches$loser_type[main_matches$loser_seeding == "(WC)"] = "wildcard"
main_matches$loser_type[main_matches$loser_seeding == "(Q)"] = "qualifier"
main_matches$loser_type[main_matches$loser_seeding == "(LL)"] = "lucky loser"

And then I cleaned up the existing column:

tidy_seeding = function(seeding) {
  no_brackets = gsub("\\(|\\)", "", seeding)
  return(gsub("WC|Q|L", NA, no_brackets))
}
 
main_matches = main_matches %>% 
  mutate(winner_seeding = as.numeric(tidy_seeding(winner_seeding)),
         loser_seeding = as.numeric(tidy_seeding(loser_seeding)))

Now we can write a query against the data frame to find out when the underdog won i.e. a player with no seeding beat a player with a seeding or a lower seeded player beat a higher seeded one:

> main_matches %>%  filter((winner_seeding > loser_seeding) | (is.na(winner_seeding) & !is.na(loser_seeding)))
                  winner winner_seeding                 loser loser_seeding                  score          round year
1          Jurgen Melzer             NA         Fabio Fognini            30           675 75 63 62   Round of 128 2013
2          Bernard Tomic             NA           Sam Querrey            21       766 763 36 26 63   Round of 128 2013
3        Feliciano Lopez             NA          Gilles Simon            19             62 64 7611   Round of 128 2013
4             Ivan Dodig             NA Philipp Kohlschreiber            16 46 676 763 63 21 (RET)   Round of 128 2013
5         Viktor Troicki             NA      Janko Tipsarevic            14              63 64 765   Round of 128 2013
6         Lleyton Hewitt             NA         Stan Wawrinka            11               64 75 63   Round of 128 2013
7           Steve Darcis             NA          Rafael Nadal             5             764 768 64   Round of 128 2013
8      Fernando Verdasco             NA      Julien Benneteau            31             761 764 64    Round of 64 2013
9           Grega Zemlja             NA       Grigor Dimitrov            29       36 764 36 64 119    Round of 64 2013
10      Adrian Mannarino             NA            John Isner            18               11 (RET)    Round of 64 2013
11         Igor Sijsling             NA          Milos Raonic            17              75 64 764    Round of 64 2013
12     Kenny De Schepper             NA           Marin Cilic            10                  (W/O)    Round of 64 2013
13        Ernests Gulbis             NA    Jo-Wilfried Tsonga             6         36 63 63 (RET)    Round of 64 2013
14     Sergiy Stakhovsky             NA         Roger Federer             3         675 765 75 765    Round of 64 2013
15          Lukasz Kubot             NA          Benoit Paire            25               61 63 64    Round of 32 2013
16     Kenny De Schepper             NA           Juan Monaco            22              64 768 64    Round of 32 2013
17        Jerzy Janowicz             24       Nicolas Almagro            15              766 63 64    Round of 32 2013
18         Andreas Seppi             23         Kei Nishikori            12        36 62 674 61 64    Round of 32 2013
19         Bernard Tomic             NA       Richard Gasquet             9          767 57 75 765    Round of 32 2013
20 Juan Martin Del Potro              8          David Ferrer             4              62 64 765 Quarter-Finals 2013
21           Andy Murray              2        Novak Djokovic             1               64 75 64         Finals 2013

There are actually very few times when a lower seeded player beat a higher seeded one but there are quite a few instances of non seeds beating seeds. We’ve got 21 occurrences of underdogs winning out of a total of 127 matches.

Let’s filter that set of rows and see which seeds lost in the first round:

> main_matches %>%  filter(round == "Round of 128" & !is.na(loser_seeding))
           winner winner_seeding                 loser loser_seeding                  score        round year
1   Jurgen Melzer             NA         Fabio Fognini            30           675 75 63 62 Round of 128 2013
2   Bernard Tomic             NA           Sam Querrey            21       766 763 36 26 63 Round of 128 2013
3 Feliciano Lopez             NA          Gilles Simon            19             62 64 7611 Round of 128 2013
4      Ivan Dodig             NA Philipp Kohlschreiber            16 46 676 763 63 21 (RET) Round of 128 2013
5  Viktor Troicki             NA      Janko Tipsarevic            14              63 64 765 Round of 128 2013
6  Lleyton Hewitt             NA         Stan Wawrinka            11               64 75 63 Round of 128 2013
7    Steve Darcis             NA          Rafael Nadal             5             764 768 64 Round of 128 2013

Rafael Nadal is the most prominent but Stan Wawrinka also lost in the first round that year which I’d forgotten about! Next let’s make the ’round’ column an ordered factor one so that we can sort matches by round:

main_matches$round = factor(main_matches$round, levels =  c("Round of 128", "Round of 64", "Round of 32", "Round of 16", "Quarter-Finals", "Semi-Finals", "Finals"))
 
> main_matches$round
...     
Levels: Round of 128 Round of 64 Round of 32 Round of 16 Quarter-Finals Semi-Finals Finals

We can now really easily work out which unseeded players went the furthest in the tournament:

> main_matches %>% filter(is.na(loser_seeding)) %>% arrange(desc(round)) %>% head(5)
             winner winner_seeding             loser loser_seeding           score          round year
1    Jerzy Janowicz             24      Lukasz Kubot            NA        75 64 64 Quarter-Finals 2013
2       Andy Murray              2 Fernando Verdasco            NA  46 36 61 64 75 Quarter-Finals 2013
3 Fernando Verdasco             NA Kenny De Schepper            NA        64 64 64    Round of 16 2013
4      Lukasz Kubot             NA  Adrian Mannarino            NA  46 63 36 63 64    Round of 16 2013
5    Jerzy Janowicz             24     Jurgen Melzer            NA 36 761 64 46 64    Round of 16 2013

Next up I thought it’d be cool to write a function which showed which round each player exited in:

round_reached = function(player, main_matches) {
  furthest_match = main_matches %>% 
    filter(winner == player | loser == player) %>% 
    arrange(desc(round)) %>% 
    head(1)  
 
    return(ifelse(furthest_match$winner == player, "Winner", as.character(furthest_match$round)))
}

Our function isn’t vectorisable – it only works if we pass in a single player at a time so we’ll have to group the data frame by player before calling it. Let’s check it works by seeing how far Andy Murray and Rafael Nadal got:

> round_reached("Rafael Nadal", main_matches)
[1] "Round of 128"
> round_reached("Andy Murray", main_matches)
[1] "Winner"

Great. What about if we try it against each of the top 8 seeds?

> rbind(main_matches %>% filter(winner_seeding %in% 1:8) %>% mutate(name = winner, seeding = winner_seeding), 
        main_matches %>% filter(loser_seeding %in% 1:8) %>% mutate(name = loser, seeding = loser_seeding)) %>%
    select(name, seeding) %>%
    distinct() %>%
    arrange(seeding) %>%
    group_by(name) %>%
    mutate(round_reached = round_reached(name, main_matches))
Source: local data frame [8 x 3]
Groups: name
 
                   name seeding  round_reached
1        Novak Djokovic       1         Finals
2           Andy Murray       2         Winner
3         Roger Federer       3    Round of 64
4          David Ferrer       4 Quarter-Finals
5          Rafael Nadal       5   Round of 128
6    Jo-Wilfried Tsonga       6    Round of 64
7         Tomas Berdych       7 Quarter-Finals
8 Juan Martin Del Potro       8    Semi-Finals

Neat. Next up I want to do a comparison between the round they reached and the round you’d expect them to get to given their seeding but that’s for the weekend!

I’ve put a CSV file containing all the data in this gist in case you want to play with it. I’m planning to scrape a few more years worth of data before Monday and add in some extra fields as well but in case I don’t get around to it the full script in this blog post is included in the gist as well so feel free to tweak it if tennis is your thing.

Categories: Blogs

How Kanban Can Help Your Team

Is your team overburdened with too much work? Are there too many conflicting priorities? Watch this webinar to find out how Kanban can help your team. About This Webinar David Neal, Developer Advocate for LeanKit, recalls his experiences getting started with Kanban, sharing stories and lessons learned. You’ll also get tips on how to: Make […]

The post How Kanban Can Help Your Team appeared first on Blog | LeanKit.

Categories: Companies

Sketch Your Way to Faster Consensus and Better Products

Agile Management Blog - VersionOne - Thu, 06/25/2015 - 14:30

This meeting is a waste of my time. When was the last time you had that thought? Was it because the conversation wasn’t focused, or people couldn’t agree, or maybe they were in violent agreement, but couldn’t see it?

We recently spoke with Jeremy Kriegel, an independent UX consultant, at Agile Day Atlanta, about a sketching technique you can use to get your meetings back on track, get to consensus faster, and deliver better products.

Jeremy-Kriegel--MSM
VersionOne: Why is sketching so important to agile teams?

Jeremy: It’s not so much about sketching per se. It’s about moving agile teams forward and getting to decisions faster. Sketching is one great way to do that, because people think in multiple ways. When you’re just talking, it’s just words and concepts, but when you add pictures, the communication becomes a lot clearer.

VersionOne: How do you respond when people say “I can’t draw”?

One of the barriers to doing sketch facilitation is that most people think they can’t draw. They’re thinking about sketching in terms of creating art. There’s a difference between sketching as art, and sketching as communication. When you’re sketching as communication, you only need some rudimentary drawing skills and a few basic techniques in order to communicate an idea and collaborate with people.

VersionOne: Are there any particular sketches or symbols that would be helpful for people to learn? Or do people just need to get over worrying about whether or not their sketches look good?

Jeremy: It’s a little bit of both. In my workshops, I start out by showing a beautiful wireframe that I found online that has crosshatching and perfectly straight lines that looks like someone took a lot of time and effort to create. Most people would agree it’s a beautiful sketch, but it takes a lot of time, and you just don’t have that kind of time in a meeting. Then I show people my version of that same wireframe, which is a really messy, squiggly, drawing, but it has all the right elements. That’s the first time people start to go, “Oh, I could do that.”

Then I start to break it down in terms of showing them a screen, like a regular web page, and say, “Okay now here’s a sketch version of that screen. Look at the elements and just draw a bar for the navigation bar and box with an x in it is an image place holder, etc. When they see it step by step, I think it starts to make sketching more comfortable.

Sketch

 

 

 

 

 

 

 

 

 

Once I’ve demonstrated a number of techniques on how to represent text, people can start with very basic sketches with a couple of squiggly lines representing a line of text or a paragraph. Then they can move on to more advanced sketches that include details on different content, like a product description or directional text like “sign up for a newsletter” or “buy our product now,” to give people a better idea of what the content is intended to be.

The bottom line is if you can draw a straight line, a circle, a squiggly line, and the alphabet, that’s really all you need in order to do sketching for communication.

The next step is to break down this barrier between what they think they can do, and being able to do it. I start with a fairly simple webpage and give people a minute to sketch that page. Then I show them more complicated page and give them a minute to sketch that page. Then I show them a mobile app, and give them a minute to sketch that. That way they get some practice sketching different types of pages and content, and they have to do it really fast.

The point of sketching quickly is not to prove how fast you can go, but if you’re trying to facilitate a group discussion, the ideas are usually coming fairly rapidly and you have to be able to keep up with the conversation. This also helps you move away from this notion of being perfect. You just don’t have time to be perfect when you’re trying to capture a lot of input quickly.

Now that they’ve copied a few pages of sketches, then I ask them to take a common page type that they’re all familiar with, typically an e-commerce check out page, and sketch that from memory. That way I can ease them into the process from seeing what’s involved and seeing some examples, to sketching a page in front of them, and then creating their own sketches. That starts to get people over this fear of sketching.

In the last exercise, I ask people to pair up. One person plays a product owner and the other person plays the graphic facilitator. The first half of the exercise the product owner says, “We’re going to complete this checkout process. Here’s what I need on my shipping page.” The sketch facilitator visually captures the input in words to make sure they have their shared understanding of what they’re going to design. He captures the words that product owner is saying and writes the words down so the product owner can see and agree to them. Once they’ve done that they’ll move on to sketching with the sketch facilitator driving with input in real time from the other person. Then they’ll switch roles and go through the exercise again.

VersionOne: Have you seen examples when a team is having a difficult time communicating and then they start sketching and everything becomes better?

Jeremy: In my almost 20 years of being in this industry, sketching is one of the most powerful tools to help move teams forward quickly.

Years ago when I was at a big agency we always kicked off projects with these big workshops with stakeholders. We always took notes on the whiteboard, so that all the stakeholders and the team members could see, and agree, on what was being discussed. If people see the input in real time you can avoid issues later. If someone disagrees with something that you’ve written, they’ll say it right away.

Sketching saves a lot of back and forth time. You can discuss conceptually what you’re looking for, and then collaboratively visualize the project.

I’ve really seen the difference when I’ve worked with other teams where either no one was taking notes or someone sent notes in a follow-up email that no one actually looked at it.

VersionOne: Do you have any advice to help people get over their fear of sketching in front of a team?

Jeremy: Many people are nervous about getting up in front of their team, and doing something new. To help them get over this fear, I suggest that they try progressive desensitization. It’s a technique that people might be familiar with if they, or someone they know, is afraid to fly. The airlines have these programs that you can go through to help you get you over that fear.

The first step for someone who is afraid of flying might be to just drive by an airport. They know they’re not going to go in and they might feel a little anxious about it, so they just drive by an airport until that feels comfortable. Then they’ll go into the ticketing area. They know they’re not going any further, and they can just go in until that’s comfortable. They might come in and leave the first couple of times and might not go in any farther. When they’re comfortable in the ticketing area, they might go through security, and go to a gate. They’ll do that over and over again until they’re comfortable. Then they eventually feel comfortable to fly.

I suggest a similar approach with sketching. The first step could be taking notes privately, just so you get a sense that you can keep up with capturing the conversation. Then when you feel like you are getting good enough at that, then get up and take notes on a whiteboard that everyone can see, but maybe you don’t try and facilitate the meeting. Once you feel comfortable with capturing the conversation it will naturally progress to facilitation. People will look to you as the leader, and you’ll be able to take on more of a facilitator role. One of my caveats is you have to be aware of the power of the pen, because it’s very easy to control what gets captured if you’re the one writing it down.

Another way to get there is to fake it. There’s research around the impact that power poses have on your state of mind. Putting yourself in a confident pose will make you feel more confident. I demonstrate this by having people sit with their legs together, their knees together; their arms crossed, and put their head and chin in their chest. I ask them to get really small and whisper to themselves, “I’m a rock star.” People giggle a little bit, because it’s funny, you don’t feel like a rock star when you’re small like that. Then I have them stand up, and put their arms in the air, or stand in a Superman pose, with their hands on their hips, lift their chest up, hold their high, and say, “I’m incompetent.” Of course everyone laughs, because again it doesn’t work. Your mind wants to mimic the position that you put your body into.

Even if you’re a little bit nervous, just walk confidently to the front of the room with your body language saying you’re going to take charge and you’re going to be responsible for the team getting something done today. Then it’ll happen.

Click here to learn more.

Categories: Companies

Slides: Frustrated? It is probably your fault

Thought Nursery - Jeffrey Fredrick - Thu, 06/25/2015 - 12:43

I’ve just delivered my talk Frustrated? It is probably your fault at Devopsdays Amsterdam. That means my slides must be finished! Here they are to download, and I’m looking forward to the video being posted later.

Categories: Blogs

Getting Retrospective Actions Done

Ben Linders - Thu, 06/25/2015 - 11:15
I sometimes hear of teams that have stopped doing retrospectives because they didn't see any improvements. When I talk with them it often turns out that they didn't have good actions coming out of the retrospectives, or that the actions weren't done and kept coming back in the retrospective. No actions leads to no improvement. Here are some suggestions on what you can do to assure that you will have actions from retrospectives that are doable and that those actions get done. Continue reading →
Categories: Blogs

Agile Quick Links #32

Notes from a Tool User - Mark Levison - Wed, 06/24/2015 - 18:48

Links of interest to Agile and Scrum

Some interesting reading for the Agile community:
Categories: Blogs

Predicting the Unpredictable is Available

Johanna Rothman - Wed, 06/24/2015 - 15:27

PredictingUnpredictable-smallI’m happy to announce that Predicting the Unpredictable: Pragmatic Approaches to Estimating Cost or Schedule is done and available. It’s available in electronic and print formats. If you need a little help explaining your estimates or how to use estimation (even #noestimate), read this book.

 

Categories: Blogs

Expo:QA Conference

Growing Agile - Wed, 06/24/2015 - 14:14
We’ve just returned from 10 days in Europe where we att […]
Categories: Companies

the stress, the sadness, and the little smile

Derick Bailey - new ThoughtStream - Wed, 06/24/2015 - 13:00

It’s been 2 months since I’ve written one of these email newsletters. A lot has happened. More than I could remember – some of it would make a highlight reel, some would probably make a gag reel, but I think most of it would be cut on the floor of the editing room in order to make this film not be rated R.

video-editing

The Stress

It’s been a rough couple of months, truthfully. I haven’t felt like writing these emails. I’ve been busy writing marketing material for the RabbitMQ For Developers product launch I just did. I’ve been stressed about problems with SignalLeaf and getting WatchMeCode fixed up in some places. I’ve put in a lot of work on my client projects, and everything else. And I announced the shut down of SignalLeaf a little over a week ago.

All of this – the good and bad – has been incredibly stressful. Writing the marketing emails for the product launch is horribly painful for me. But apparently I did well. I sold a record $ for a product launch for me, bringing in more than $6,500. But it was stressful, and it was right in the middle of this that I decided to shut down SignalLeaf.

Making the decision was bad enough. If you’re not listening to Entreprogrammers, check out episode 69. You can hear the stress in my voice, and the anger and frustration when I make the final call to kill it.

The Sadness

It really hit me a few days ago, that I was shutting things down forever with SignalLeaf. Sure, I had written the blog post and talked about it on the podcast, and announced it to my customers. But just this last weekend, I recorded a short screencast where I showed people how to migrate from SignalLeaf over to another hosting provider. And to do this, I didn’t use some random demo account and podcast. I migrated my actual podcast, the Entreprogrammers, over to Libsyn.com – live migration, recorded for all the world to see. I stopped using my own service, on camera, in favor of someone else’s service.

And I could barely handle it, after that. I nearly shut down, personally, as well as professionally that afternoon. I took a nap. I ignored my computer.  I couldn’t bring myself to do the things that I needed to do that day, because it was too much for me to deal with. It makes me incredibly sad, right now, talk about this and relive the experience of recording and uploading that screencast.

The Right Decision

I don’t like shutting down SignalLeaf. In spite of the stress and expense that it has caused me in the last few years, I didn’t want to do it. But I did it anyways, because it was the right thing to do… for me, and for my customers. I was never able to make time to work on SignalLeaf the way I needed to. I couldn’t finish features I started. I couldn’t support people the way I told them I would. I was leading people on, while panicking about doing exactly that.

Closing SignalLeaf was the most difficult decision I’ve had to make, in a very long time… and, yet …

The Little Smile

For the last 2+ years, I’ve battled with time management. I’ve always been stressed and worried about one product or service while working on the other. Every moment that I put into WatchMeCode felt like a betrayal of my customers and service at SignalLeaf. Every time I planned work for WatchMeCode, I would panic knowing that I was putting off SignalLeaf work once again.

This last Sunday was no different. I was thinking about WatchMeCode work, and I was panicked. I was ignoring… but, I just shut down SignalLeaf, didn’t I? I… I don’t have to worry about it anymore? I don’t have to panic about ignoring it in favor of WatchMeCode? … The realization of exactly what I was doing, why I was doing it, and the reality that I now find myself within – it was a burden lifted from my shoulders.

And for the first time in I can’t remember how long, I smiled a little at the thought of spending all of Monday working on WatchMeCode.

– Derick

 

Categories: Blogs

R: Scraping the release dates of github projects

Mark Needham - Wed, 06/24/2015 - 00:34

Continuing on from my blog post about scraping Neo4j’s release dates I thought it’d be even more interesting to chart the release dates of some github projects.

In theory the release dates should be accessible through the github API but the few that I looked at weren’t returning any data so I scraped the data together.

We’ll be using rvest again and I first wrote the following function to extract the release versions and dates from a single page:

library(dplyr)
library(rvest)
 
process_page = function(releases, session) {
  rows = session %>% html_nodes("ul.release-timeline-tags li")
 
  for(row in rows) {
    date = row %>% html_node("span.date")
    version = row %>% html_node("div.tag-info a")
 
    if(!is.null(version) && !is.null(date)) {
      date = date %>% html_text() %>% str_trim()
      version = version %>% html_text() %>% str_trim()
      releases = rbind(releases, data.frame(date = date, version = version))
    }  
  }
  return(releases)
}

Let’s try it out on the Cassandra release page and see what it comes back with:

> r = process_page(data.frame(), html_session("https://github.com/apache/cassandra/releases"))
> r
           date               version
1  Jun 22, 2015       cassandra-2.1.7
2  Jun 22, 2015      cassandra-2.0.16
3   Jun 8, 2015       cassandra-2.1.6
4   Jun 8, 2015   cassandra-2.2.0-rc1
5  May 19, 2015 cassandra-2.2.0-beta1
6  May 18, 2015      cassandra-2.0.15
7  Apr 29, 2015       cassandra-2.1.5
8   Apr 1, 2015      cassandra-2.0.14
9   Apr 1, 2015       cassandra-2.1.4
10 Mar 16, 2015      cassandra-2.0.13

That works pretty well but it’s only one page! To get all the pages we can use the follow_link function to follow the ‘Next’ link until there aren’t anymore pages to process.

We end up with the following function to do this:

find_all_releases = function(starting_page) {
  s = html_session(starting_page)
  releases = data.frame()
 
  next_page = TRUE
  while(next_page) {
    possibleError = tryCatch({  
      releases = process_page(releases, s)
      s = s %>% follow_link("Next") 
    }, error = function(e) { e })
 
    if(inherits(possibleError, "error")){
      next_page = FALSE
    }
  }
  return(releases)
}

Let’s try it out starting from the Cassandra page:

> cassandra = find_all_releases("https://github.com/apache/cassandra/releases")
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-2.0.13
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-2.0.10
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-2.0.8
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.2.13
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-2.0.0-rc1
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.2.3
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.2.0-beta2
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.0.10
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.0.6
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-1.0.0-rc2
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.7.7
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.7.4
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.7.0-rc3
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.6.4
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.5.0-rc3
Navigating to https://github.com/apache/cassandra/releases?after=cassandra-0.4.0-final
 
> cassandra %>% sample_n(10)
            date               version
151 Mar 13, 2010   cassandra-0.5.0-rc2
25   Jul 3, 2014      cassandra-1.2.18
51  Jul 27, 2013       cassandra-1.2.8
21  Aug 19, 2014   cassandra-2.1.0-rc6
73  Sep 24, 2012 cassandra-1.2.0-beta1
158 Mar 13, 2010   cassandra-0.4.0-rc2
113 May 20, 2011     cassandra-0.7.6-2
15  Oct 24, 2014       cassandra-2.1.1
103 Sep 15, 2011 cassandra-1.0.0-beta1
93  Nov 29, 2011       cassandra-1.0.4

I want to plot when the different releases happened in time and in order to do that we need to create an extra column containing the ‘release series’ which we can do with the following transformation:

series = function(version) {
  parts = strsplit(as.character(version), "\\.")  
  return(unlist(lapply(parts, function(p) paste(p %>% unlist %>% head(2), collapse = "."))))  
}
 
bySeries = cassandra %>%
  mutate(date2 = mdy(date), series = series(version),
         short_version = gsub("cassandra-", "", version),
         short_series = series(short_version))
 
> bySeries %>% sample_n(10)
            date               version      date2        series short_version short_series
3    Jun 8, 2015       cassandra-2.1.6 2015-06-08 cassandra-2.1         2.1.6          2.1
161 Mar 13, 2010 cassandra-0.4.0-beta1 2010-03-13 cassandra-0.4   0.4.0-beta1          0.4
62  Feb 15, 2013      cassandra-1.1.10 2013-02-15 cassandra-1.1        1.1.10          1.1
153 Mar 13, 2010 cassandra-0.5.0-beta2 2010-03-13 cassandra-0.5   0.5.0-beta2          0.5
37   Feb 7, 2014       cassandra-2.0.5 2014-02-07 cassandra-2.0         2.0.5          2.0
36   Feb 7, 2014      cassandra-1.2.15 2014-02-07 cassandra-1.2        1.2.15          1.2
29   Jun 2, 2014   cassandra-2.1.0-rc1 2014-06-02 cassandra-2.1     2.1.0-rc1          2.1
21  Aug 19, 2014   cassandra-2.1.0-rc6 2014-08-19 cassandra-2.1     2.1.0-rc6          2.1
123 Feb 16, 2011       cassandra-0.7.2 2011-02-16 cassandra-0.7         0.7.2          0.7
135  Nov 1, 2010 cassandra-0.7.0-beta3 2010-11-01 cassandra-0.7   0.7.0-beta3          0.7

Now let’s plot those releases and see what we get:

ggplot(aes(x = date2, y = short_series), 
       data = bySeries %>% filter(!grepl("beta|rc", short_version))) +     
  geom_text(aes(label=short_version),hjust=0.5, vjust=0.5, size = 4, angle = 90) + 
  theme_bw()

2015 06 23 22 59 19

An interesting thing we can see from this visualisation is what overlap the various series of versions have. Most of the time there are only two series of versions overlapping but the 1.2, 2.0 and 2.1 series all overlap which is unusual.

In this chart we excluded all beta and RC versions. Let’s bring those back in and just show the last 3 versions:

ggplot(aes(x = date2, y = short_series), 
       data = bySeries %>% filter(grepl("2\\.[012]\\.|1\\.2\\.", short_version))) +     
  geom_text(aes(label=short_version),hjust=0.5, vjust=0.5, size = 4, angle = 90) + 
  theme_bw()

2015 06 23 23 08 04

From this chart it’s clearer that the 2.0 and 2.1 series have recent releases so there will probably be three overlapping versions when the 2.2 series is released as well.

The chart is still a bit cluttered although less than before. I’m not sure of a better way of visualising this type of data so if you have any ideas do let me know!

Categories: Blogs

CIO Magazine – Introduction to SAFe

Agile Product Owner - Tue, 06/23/2015 - 19:43

As the Scaled Agile Framework (SAFe®) expands in Fortune 1000 companies, we are starting to see more press coverage. In addition to the recently featured Forbes article, SAFe and is now getting more “C-Suite” attention, as exhibited by last weeks publication in CIO MagazineIntroducing the Scaled Agile Framework by Matt Heusser, Managing Consultant at Excelon Development. 

Such articles are building greater awareness amongst the important SAFe constituency of executives, managers and leaders, whose involvement and leadership is critical for the success of Lean-Agile adoption at enterprise scale.  This awareness will help open opportunities for internal change agents and SPCs to have a further conversation with CIOs, CTOs, IT executives and business leaders about the business benefits of implementing SAFe.

Let me know what you think about the articles and join the conversation for CIOs online at http://bit.ly/1fkeKWD

Always Be SAFe,
Richard

Categories: Blogs

The Emperor Has No Clothes: A Theory of Transformation

Leading Agile - Mike Cottmeyer - Tue, 06/23/2015 - 19:00

From Wikipedia:

“The Emperor’s New Clothes” is a short tale by Hans Christian Andersen about two weavers who promise an Emperor a new suit of clothes that is invisible to those who are unfit for their positions, stupid, or incompetent. When the Emperor parades before his subjects in his new clothes, no one dares to say that he doesn’t see any suit of clothes until a child cries out, “But he isn’t wearing anything at all!”

Something has been nagging at me for a while. You’ll see shades of it underlying the themes in my last several blog posts. You’ll see it embodied in the messaging on our website. You’ll see it in the very fabric of our company and how we approach our work.

It’s this notion that if agile doesn’t work it’s somehow your fault.

Everything we read and hear about agile is that we are supposed to empower people, let them self-organize, that the people closest to the work are best at deciding how to do the work. If only us managers would get out of the way, that would solve everything.

That sounds great… until it doesn’t work.

What’s the story told to us when it doesn’t work? We are told that maybe we just didn’t get it? That we weren’t agile enough? Maybe it’s that our culture is broken or that our leadership is too command and control? If we just had the right mindset, all would be right with the world? Call me when you’re actually ready to change. Right?

Do we ever question if self-organization really works?

Everyone has all these great success stories with agile. If I’m the guy that says agile doesn’t work, I must be the one that doesn’t get it? Maybe the culture I created is broken? Maybe I’m the one who is too command and control? Maybe it’s me that doesn’t have the right mindset? Maybe… but, I think there is more to the story.

It dawned on me the other day ‘the emperor has no clothes’.

Self-organization works in the context of small, cross-functional teams, teams that have clarity of purpose, teams that have everything and everyone necessary to deliver a working tested increment of the product on regular intervals. It works when the team has the autonomy to decide.

The question is this… will people self-organize into teams and systems that support this kind of delivery model? There are many things about this kind of system that are counter-intuitive. They definitely go against how people are used to working. Maybe someday… but probably not right out of the gate… at least not enough of them.

Many of us are selling something that just isn’t true.

I don’t mean that in an ugly way… I think many of us have ultimate faith in human potential and want to believe this will work. I think that’s good. I might even allow that in certain contexts, under the right conditions, fully self-organizing systems actually could work. The problem is that it doesn’t work in enough contexts enough of the time. I’ve seen too many data points to the contrary.

Let’s just say it… the emperor has no clothes

The problem is that many folks think very linearly. Many folks are married to old beliefs. Many folks are not systems thinkers. Many folks want to locally optimize. Some of us bring personal agendas, individual goals. Some of us are narcissistic sociopaths. Some are just scared to change.

It’s not that we are all bad people… we just think differently.

I do not believe that self-organization will work in organizations when there is misalignment between the technology organization and the business. I do not believe that self-organization will trump legacy financial controls and crushing dependencies between systems. The forces working against change are too great.

Because we believe so deeply that self-organization works. Because it’s not popular to think that anything less than total empowerment of the individual could possibly make a difference or be of value… because we don’t want to be accused of being ‘command and control’… we do nothing.

Revolution or evolution… it’s your call

The real problems facing companies today can’t be solved by self-organization alone. We are talking about a major architectural refactoring in many companies. We have to address business architecture… technology architecture… organizational architecture. We have to create alignment between these three.

We have to redesign how work get’s funded. We have to redesign how commitments are made. We have to redesign how we make economic tradeoffs in the face of limited resources to do the work… all while we manage to stay in business. These are not cultural problems, they are systems problems.

Most of us don’t operate with a fundamental theory of how organizations are supposed to work. How can we be expected to self-organize this kind of system?

Rest assured, these are solvable problems… but first you have to buy into the nature of the problem.

If we define it as a culture problem or a process problem… we fail.

If we recognize it as a systems problem… we have a shot at fixing it.

In the absence of simply telling folks to self-organize, where do you start?

Understand where you are and where you want to go

I’ve been speaking publicly the last year or so around a model we created to help companies sort out their goals and objectives, to help them decide what they want to be as a company and to understand where their markets want them to be.

We call this model ’The LeadingAgile Compass’

Here is a link to the full explanation of the compass:
http://www.leadingagile.com/our-compass

We’ve also been using the metaphor of expeditions and basecamps to explain the incremental and iterative nature of transformation. Basically you start with a subset of the organization, get it working quickly in an agile framework, and improve agility over time. Wash, rinse, repeat.

We call this model ‘The LeadingAgile Journey’

Here is a link to the full explanation of the LeadingAgile Journey:
http://www.leadingagile.com/the-journey

Finally, we’ve been articulating an execution strategy that starts with defining a structure, governance, and metrics framework for the organization… piloting that framework on a single expedition, taking that expedition to the first basecamp, and then broadening the transformation to the larger enterprise.

We call this model ‘The LeadingAgile Roadmap’

Here is a link to the full explanation of the LeadingAgile Roadmap:
http://www.leadingagile.com/the-roadmap

Taken together, these models articulate a strategy for planning, executing, and controlling an agile transformation. The hard, cold reality is that many companies will never become more agile unless we can demonstrate early BUSINESS FOCUSED results and a plan for continuous improvement.

The big win here, is that the teams achieve a fairly high level of agility right out of the gate. They get better using team-based, iterative and incremental techniques, while learning the basics of agile early in the process. The organization will quickly stabilize and become predictable.

As your overall understanding of the underlying mechanisms of agility matures, and real impediments are removed, as dependencies are broken, business agility will improve across the entire enterprise.

Does this necessarily preclude self-organization?

Not really. The actual implementation of this transformation framework isn’t dictated by a few consultants or a few executives from some smoky conference room somewhere, it’s done collaboratively with the people involved in the change.

Maybe all we are doing here is giving structure and guidance for self-organization to take place. Maybe.

For me… implementing agile is about forming teams, defining backlogs, and producing increments of working tested products on regular intervals. Breaking dependencies over time is key. Anything that get’s in the way of that is an impediment. Period.

The patterns are well understood, implementation of those patterns is tough. Non-trivial problems I like to say.

In Conclusion…

Teaching people the basics of Scrum and telling them to self-organize, except in possibly the smallest of organizations, is irresponsible. I think most companies will need more guidance.

Creating the preconditions for agile to thrive is a science that can be understood, planned, executed and measured… even if you let your people self-organize around it. It’s not magic.

Agile fails because the underlying preconditions to do agile well are not clear. You might not have the right mindset, and maybe you are a little too command and control, but that’s not your biggest problem.

Don’t be afraid to speak up when the emperor isn’t wearing any clothes.

The post The Emperor Has No Clothes: A Theory of Transformation appeared first on LeadingAgile.

Categories: Blogs

ScrumDesk Start! 3.8 Released

Scrum Expert - Tue, 06/23/2015 - 17:43
ScrumDesk has released ScrumDesk Start! v.3.8 that provides print, filter and mentions. ScrumDesk Start! is a free Scrum board for small Agile teams focused on getting the results instead of complicated process. It brings core Scrum practices fundamental for fast product development by self-organized teams. Finally printing has appeared in ScrumDesk Start! As we have great planning experience with physical cards, we decided for card layout that we suggest to clients of our consultancy team. Printed card layout provides fundamental information necessary for proper release or sprint planning. Other improvements Advanced filters Even all ...
Categories: Communities

Knowledge Sharing


SpiraTeam is a agile application lifecycle management (ALM) system designed specifically for methodologies such as scrum, XP and Kanban.