Skip to content

Feed aggregator

Links for 2016-08-24 []

Zachariah Young - Thu, 08/25/2016 - 09:00
Categories: Blogs

Quotable Quotes: Limit Work-In-Progress As Much As Possible!

Learn more about transforming people, process and culture with the Real Agility Program

Jerry Doucett 201601 white background - head - 275 squareScrum team members should be allocated to as few different initiatives as realistically possible.  The more projects you are allocated to, the more task switching you may have to perform, the longer it will take to complete any one item, the thinner you will be spread and the less effective you will be.  In other words, people (and teams) should limit their work in progress as much as possible and focus on completing those things that truly matter most.  This is true for any framework, but it is particularly emphasized with Agile ones.  Note there is a slight but fundamental difference between being allocatedto a team and being dedicated to a team – that is a topic for a future article.

(By Senior Agile Coach Jerry Doucett)


Jerry is leading a series of SAFe training classes in Toronto, Ontario from September through to December 2016. See here for more details.

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Quotable Quotes: Limit Work-In-Progress As Much As Possible! appeared first on Agile Advice.

Categories: Blogs

Principle-Driven Design

Kanbanery - 6 hours 58 min ago

Artykuł Principle-Driven Design pochodzi z serwisu Kanbanery.

Whether you’re struggling to improve your product, juggling too many conflicting feature requests, or facing growing levels of competition, a values-based approach to looking at those problems may provide the solution you need.

Values-based Product Design

Design ideas can come from anywhere – user research, ideation, a shower epiphany, brainstorming, a dream. You won’t implement most of them, luckily, but that doesn’t mean that they aren’t useful. Why and how we reject ideas becomes an important part of company culture. It also is a building block of brand personality and character. And after all, the product itself is just an artifact of the company culture.

From the moment Kanbanery was live, I’ve been getting feature requests. My relationship with feature requests has changed dramatically over the years. In the early days, when there were just five of us building frantically and feeling delighted to have our first hundred, then thousand, and then ten thousand users, I thought of feature requests as work orders. Stuff I had to do to keep those people happy. Now I hear them only as cries for understanding that I can often address without writing a single line of code. Often there are non-technical or third-party solutions. Sometimes all they want is a bit of sympathy.

As Mike Burrows illustrates so well in his book, blog, and conference talk, the Kanban Method rests on a set of core values. The biggest barrier to implementing a Kanban system is when the values inherent to the tool clash with the values of the company or team using it. How is a secretive team that doesn’t trust others supposed to make all their work visible? What attracted me to Kanban in the first place was that it meshed so well with my values. As the founder and president of Lunar Logic, I’d built a company that reflected my values. I had employees who either shared those values or at least functioned well in an environment modeled after them.

Systems Thinking

How do values impact product decisions? The easiest answer is to share a few examples. A frequent feature request is to add employee-level metrics. “I want to see who is putting in the most time on tasks” or a common request is “show me cycle time by person.” I’ve always refused and replied that Kanban is a tool for improving systems, not for stack-ranking employees. Improvements should be implemented at the system level, not at the unit level (look for a future post from me “I don’t believe in lazy people”). Systems thinking (or holism, as it’s called in my first discipline of anthropology) is important enough to me to lose a client over. I’ve resisted any feature that could be used to blame or judge an individual. That same value has led me to reject custom board views for individual users. A team can’t collaborate on improving a system if they don’t share the same view of it.

Respect for People

Another core principle Kanban and I is respect for people. How does this value express itself in Kanbanery? I respect people’s time, so our onboarding emails are as concise as possible, with no marketing fluff. The estimated reading time is always under two minutes, and I include it in the subject line. I send it from my personal email address, so your replies come straight to me.

We would like people to visit their Kanbanery board often, but we won’t trick them into doing it like so many other SaaS products. When we send a notification about a chat message we include the full text of the message. I hate getting emails that read something like “You’ve got a message waiting. Log in to see it.” That feels rude to me. If you’ve got a message for me, give it to me, and let me decide if it’s worth logging in to react to it now or not.


I don’t believe in technical solutions to human problems. If a manager doesn’t trust her team, the solution isn’t to place limits on what they can do. Doing that violates my principle of trust in people. That’s why I reject requests to create rules that only let certain people pull tasks from a column, or change the owner of a task. A physical board works with no such rules. There’s no way to keep someone from walking over to a wall and ripping up a Post-it note, moving tasks backward, or changing an estimate. To place such rules in Kanbanery would be like agreeing that it’s okay not to trust your team. It’s not. That’s a problem, and the solution isn’t more control.


I value transparency. I think it’s a core component of creating trust and essential for effective collaboration. When I was running an offshore web development company, transparency was my primary answer to everything. I remember the first time one of my employees made a mistake and wasted a day’s work. He asked me what he should tell the team (including the client) during the stand-up meeting. “The truth,” I said. A client will never find a team of people who never makes a mistake, but a team that will tell them the truth, all the time, even when it’s embarrassing? That’s priceless. And that’s why in Kanbanery it’s impossible to hide content from anyone who has access to a board. People have asked for it, but I won’t do it.

That’s also why Kanbanery is the only Kanban application that lets people download the raw data we use for metrics. A metric is worse than useless if you don’t trust it. So we share the data and the calculations so that people can import it into a spreadsheet and test it themselves, or tweak it and form their own conclusions.

Values-based Marketing

It was a few years into the life of Kanbanery before I realized how those values were shaping the product, and a few years more before the opportunities that approach presented became clear.

Programmers make many small decisions in the implementation of a feature that aren’t part of the specification. Many of them are never discussed with a product owner or product manager. Some only show up in edge cases, so they may not even get tested. At the team level, being clear about a product’s principles helps everyone to make better and more consistent decisions.

It also makes it a lot easier to explain your choices to users. When I reject a feature request on the basis of one of our values, some clients understand and consider changing how they think about work. Even when they don’t share my values, they always respect my decision and appreciate my explanation.

But there’s another huge value to principle-driven development that I’m just beginning to see myself. When we built Kanbanery, we were doing it for ourselves and people like ourselves. It was a tool for software teams, and especially distributed teams like ourselves. There are millions of them, and if they wanted an online Kanban board, they had to choose between three options, all built and released in the same year. The tools were different enough that we could all share the market with plenty of room to grow. But coding a Kanban application isn’t rocket science. A single programmer can make a basic online Kanban board in a day. And so now, eight years later, I can’t count the number of competitors Kanbanery has, and more are always emerging. If we’re all targeting distributed software teams, that pie starts looking smaller, as does the distinction between the competition.

Redefining Market Demographics

But what if we change how we define our market? What if Kanbanery is a productivity tool for systems thinkers who value transparency, trust, and believe in treating people like adults? That’s not only a much bigger market, but there’s no one else targeting it.

Now almost half of the people I’ve spoken to who use Kanbanery aren’t in IT at all. They’re songwriters, professors, construction workers, bankers, dentists, and hospital administrators. They’re discovering innovative ways to use Kanbanery that I wouldn’t have imagined. What does a dentist in Maine have in common with a project manager in Bangalore. Perhaps they both found the right tool for them because they share similar values.

By designing a tool based on principles and the metaphor of a physical board, we inadvertently created something as universal as the Post-it note or whiteboard. That’s much more powerful, and marketable than the ideal tool for distributed software development.

Artykuł Principle-Driven Design pochodzi z serwisu Kanbanery.

Categories: Companies

Interview: Garry Berteig Reflects on Being Agile In a Crisis

Learn more about transforming people, process and culture with the Real Agility Program

FortMcMurrayEvacuationOn May 03, 2016 wildfires encompassing Fort McMurray, Alberta forced the evacuation of more than 88,000 residents, including many friends, family and associates of BERTEIG.

Photo Credit: Wikipedia

One such resident was Garry Berteig, a co-founder of OpenAgile, and long-time resident of Fort McMurray.

Recently, I had the opportunity to speak with Garry from his home in northern Alberta. He presents some meaningful insights into how living and working with an agile mindset helped him and his family move through the disaster with stability. The following articles shares, in his words, some highlights of his experience during and after the event.


By Senior Agile Coach Garry Berteig

Garry portrait

Most of the time, when you are acting with agility you are sort of ready for change or turn in direction anyhow. You know what you have to do so your tasks are already in motion, your tasks are already moving towards “done”, not that you have your suitcase packed already but you already know what is the most important thing.

It’s the same for corporations. It’s just not so direct or life-threatening as this catastrophe was but for some companies catastrophe is a slow burn.

The actuality about what occurred, was quite different than the media reports. The media reports were at best simple, at worst a high-level notion.

Yes there was a line of cars, and they show a line up of cars [in the news] but it doesn’t tell you that some people were in one way or another in a state of calm because they are used to being involved in safety measure. That group was relatively calm.

There’s another set of people who were actually terrified and not able to be rational so they have to be handled carefully.

Then another group of people who were on the opposite extreme were kind of ‘having a good time.’ It was taking them away from their normal routines so they were making light of the whole situation. They were rolling windows down, playing loud music, giving peace-signs, stuff like that.

The real take-home point, having landed in Edmonton and being involved with people here is that the kindness, hospitality, generousity and sympathy that people revealed was amazing.

This is a feature that I share and have been sharing with other people who have also found the same thing. It’s been quite remarkable. Their private lives had an opportunity present itself openly. In my opinion, it represents a spiritual condition. Usually people hold that in to themselves and have no opportunity to express that. [They want to be kind, generous and helpful but keep it internalized.]

The government and non-profits have also been incredibly efficient and helpful. Because of all the other experiences with other catastrophes, their quick response to 90,000 people leaving Fort McMurray was remarkable. Within one week they had arranged for financial support for all those people. That to me is the material future of Canada. It is positive. The material and spiritual future of Canada is very great. And that is what I’ve seen here.

The other point is that before Fort McMurray was black-listed by the media but that has turned 180 degrees. Now people will realize the participation in this city from all over the country. The nation raised 1 million through the Red Cross.

Thankfully, not only is Garry and his family safe and well but all other friends and associates are also settling back into their routines and beginning the long journey ahead of restoration and recovery.


UPDATE: August 03, 2016 – Unbelievably, epic flooding has now hit Fort McMurray, and in places the flood waters are damaging houses which had survived the fire. More updates will be shared as they emerge. Garry’s family is still doing well.


Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Interview: Garry Berteig Reflects on Being Agile In a Crisis appeared first on Agile Advice.

Categories: Blogs

New Case Study: Amdocs cuts idea-to-production time by almost half after moving to SAFe

Agile Product Owner - 9 hours 51 min ago

We reduced the time to take an idea to production, and our time-to-value has gone down using the SAFe® process. If reaching production would normally take 1½ years, now it could be eight months with the new processes and approach.
—Hrishikesh Karekar, Lead Agile Coach

case-study-box-amdocsOur latest case study comes to us from the Israel office of global IT powerhouse, Amdocs, where a team applied some creative tactics to take Agile successfully from theory to a successful rollout.

Amdocs selected SAFe because they needed to roll out Agile practices quickly at various levels of the organization, and the Framework provided the most robust guidance. Unlike previous efforts at applying Agile methods, this time the company followed SAFe best practices closely, and relied on SAFe-certified Agile coaches to guide the execution.

What’s  cool about this implementation was the team’s use of gamification to increase adoption at the Team level. They created a game that required teams to check Scrum items off a list, with teams earning points based on how quickly they finished their checklists. As teams completed their tasks, they took selfies and posted them on physical boards with their marked checklists. The game drove deeper understanding of the new process and more discipline in adhering to it.

This all resulted in some impressive outcomes that proved to the team that Agile is no longer theory at Amdocs:

  • Shortened time from idea to production from 1.5 years to 8 months
  • More frequent deliveries to production
  • 30% faster delivery for user acceptance testing
  • Demos every 2 weeks to 2 months, instead of 4 months
  • System stabilization time down from 6 weeks to less than a week

Most importantly, customers noticed and commented, inspiring the organizations to run more initiatives through SAFe, and management to encourage a company-wide rollout of the Framework.

You can link to the full case study here to get the rest of the story.

Many thanks to Levana Barkai, Amdocs’ Customer Delivery Manager, and Hrishikesh Karekar, Lead Agile Coach, for sharing their SAFe journey.

Stay SAFe,










Categories: Blogs

Integreer Kwaliteit met Lean Software Ontwikkeling

Ben Linders - 12 hours 1 min ago

Integreer kwaliteit Agile LeanAgile methoden zoals Scrum leggen de nadruk op functionaliteit. Klanten verwachten echter naast functionaliteit dat ook de kwaliteit van het product in orde is. Waar Agile voornamelijk aandacht geeft aan het software team en de interactie met de omgeving, kijkt Lean naar de gehele keten: van klantbehoefte tot waarde voor de klant. Een van de aspecten van Lean Software Ontwikkeling is het integreren van kwaliteit.

Lean Software Ontwikkeling combineert Agile en Lean met de volgende 7 principes:

  1. Verminder Verspillingen (Eliminate Waste)
  2. Integreer Kwaliteit (Build Quality In)
  3. Leer Voortdurend (Learn Constantly)
  4. Lever Snel (Deliver Fast)
  5. Betrek Iedereen (Engage Everyone)
  6. Verbeter Continue (Keep getting Better)
  7. Optimaliseer het Geheel (Optimize the whole)
Kwaliteit begint bij de klanten


Retrospectives Exercises Toolbox - Design your own valuable Retrospectives

Mijn definitie van kwaliteit (zie Hoe zorg je met Scrum voor Kwaliteitsproducten en Diensten) is:

Kwaliteit is de mate waarin voldaan wordt aan de behoeften van de gebruikers, en aan de eisen van de opdrachtgevers. Dat kunnen zowel functionele behoeften zijn (iets wat het product of dienst moet doen), of “performance”of niet-functionele eisen (hoe snel, hoe veel, de betrouwbaarheid, etc), vaak is het een combinatie van beide.

Het zijn je klanten die bepalen wat kwaliteit is (en wat niet), en wat vereist is voor een goede kwaliteit. Pas als je weet wat je klanten nodig hebben kun je naar goed oplossingen zoeken om aan hun wensen te voldoen. Je integreert daarmee kwaliteit in het gehele ontwikkelproces. Het juiste product

Hoe kom je erachter wat de klant nodig heeft? Agile kent diverse practices waarin het team en de klanten (in de Scrum praat men over product owner ipv klant) intensief samenwerken om ervoor te zorgen dat het juiste product geleverd wordt. Voorbeelden daarvan zijn de planning game en de product demonstratie.

In de planning game gaat het erom dat de product eisen duidelijker worden. Wat willen klanten bereiken met het product, welke waarde moet het hun gaan bieden? Wat moet het product doen om die waarde te kunnen leveren? Maar ook de kwaliteitseisen: hoe snel moet het werken, en hoe betrouwbaar en stabiel moet het zijn? Het team benut zijn kennis en ervaring om te bepalen wat mogelijk is, en hoe het product op een Lean manier ontwikkelt kan worden.

In de product demonstratie laat je het product zien aan de klanten en vraag om je feedback. Is dit wat de klant wil? Is het goed genoeg? Maar ook: Is het snel genoeg, handig te gebruiken. Betrouwbaar en veilig? En, gegeven wat het product nu doet, wat is er nog meer nodig? Kwaliteit met Agile en Lean

Hoe gaat dat in de praktijk? Laten we kijken naar een medisch systeem wat specialisten gebruiken om onderzoek te doen met patiënten. De specialisten (klanten) willen een aantal mogelijkheden hebben om data en scans van een patiënt te bekijken. Ze willen beelden van eerdere scans kunnen oproepen, inzoomen, en vergelijken. Omdat ze dat vaak met de patiënt erbij doen moet dat snel gaan, en eenvoudig te bedienen zijn. De gegevens mogen niet fout zijn, de specialisten gebruiken ze om beslissingen te nemen waar het leven van de patiënt van af kan hangen.

In de discussies vooraf en in de planning game formuleren de product owner en het team de acceptatie criteria. Voor kwaliteitseisen moeten die criteria meetbaar zijn. Dus niet “snel genoeg” maar “in 90% van de gevallen reageert het systeem binnen 1 seconde”.

Samen met de product owner formuleren de teamleden user stories. De acceptatiecriteria in de user stories worden door het team gebruikt om af te spreken hoe ze de software gaan maken en verifiëren.

Bijvoorbeeld, voor een bepaalde user story doen ze een spike, ze maken een stukje sofware en een testcase die meet hoe snel de software is om te bepalen of wat de klant wil haalbaar is. Voor een andere user story wil het team pair programming gebruiken, het is een complexe functie waarmee de team leden nog geen ervaring hebben.

Er zijn ook stories waarbij test driven design volgens het team de beste aanpak is, en een enkele story waarbij de klant nog niet echt weet wat het product precies moet gaan doen om er op  een handige manier mee te kunnen werken, daar lijkt prototyping met Lean Startup het beste te passen.

De vereiste functionaliteit en kwaliteit is bepalend voor de aanpak. De product owner maakt duidelijk wat er nodig is en welke kwaliteit de klanten verwachten. Het team weet wat met een bepaalde manier van werken haalbaar is, en check in de planning game met de product owner. Te weinig kwaliteit is niet goed, maar teveel ook niet. Het gaat bij lean om het vinden van de juiste balans tussen tijd, geld en kwaliteit voor het leveren van functionaliteit.

In de demonstratie wordt de software getoond en gechecked of het voldoet. Daarbij telt zowel de functionaliteit als de kwaliteit. Het moet niet alleen werken, het moet ook snel genoeg zijn, betrouwbaar, bedienbaar, etc. Pas dan voldoet het product aan alle eisen en is het af.

De kracht zit in de samenwerking tussen het team en de product owner gedurende de ontwikkeling. Is er een gedeeld beeld wanneer, hoe en waarvoor klanten het product gebruiken? Wat betekent het product voor hun en welke waarde het kan toevoegen? Kan de product owner voldoende duidelijk maken wat nodig is, en checken de teamleden of ze het goed begrepen hebben? Leren ze van dingen die niet goed zijn gegaan? Integreer kwaliteit

Lean en Agile versterken elkaar als het gaat om de kwaliteit van de producten en diensten. Met agile en lean practices integreer je kwaliteit in de volledige productontwikkelingsketen.

kwaliteitsverbeteringIn de workshop software kwaliteitsverbetering leer je hoe je goede producten en diensten kunt leveren. Kwaliteitsverbetering helpt organisaties om beter te voldoen aan de behoeften van de gebruikers en aan de eisen van de opdrachtgevers.

Categories: Blogs

Case Study: ATDD Lets Team Frequently Release Software

NetObjectives - 13 hours 59 min ago
This is a report from a team at a leading financial company: Our project team was formed to deliver a business critical function with a fixed delivery date. Many of the team members were new to line of business that we were asked to support. Early in the project, Acceptance Test-Driven Development, and more specifically Cucumber, was a useful tool for capturing and documenting the expectations of...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]
Categories: Companies

Mentions and Attachments for Android: Release 1.7

TargetProcess - Edge of Chaos Blog - 14 hours 15 min ago

Our latest Android release is going to make communicating with colleagues a lot easier. You are now able to mention people in comments and share photos from your device.


When you’re writing a comment, just use the @ symbol, followed by the name of the person you want to notify. They’ll receive an email notification a few seconds after the comment is submitted.


Previously, you were able to view an entity’s attachments and photos but could not add new ones. Now, you can.
Open the Attachments tab and choose a way to add pictures: select an existing one from your device, or take a new one with your camera.

We’re working on improving navigation and general ease of use for our next Android release. Until then, feel free to let us know if you have any feedback or comments to share with our team – just shoot us an email at

Categories: Companies

Continue Verbetering met Agile in Bits&Chips

Ben Linders - 14 hours 22 min ago

bitchipslogoMijn artikel Continue Verbetering met Agile is gepubliceerd in Bits&Chips nr 4. In dit artikel laat ik zien dat continue verbetering een integraal onderdeel is van de Agile-mindset en van de Agile-principes en -practices, en geef ik tips en advies voor verbeteren met agile:

Silver bullets bestaan niet in softwareontwikkeling. Effectieve softwareteams bepalen zelf hoe ze hun werk doen, passen zich continu aan en verbeteren zichzelf. Continue verbetering is ingebed in de Agile-mindset en -principes en helpt zo om de flexibiliteit in bedrijven te verhogen en meer waarde te leveren, betoogt Ben Linders.


Retrospectives Exercises Toolbox - Design your own valuable Retrospectives

Bits&Chips publiceert een magazine over ontwikkelingen in de high-tech industrie en organiseert de jaarlijkse smart systems conferenties (Software-Centric Systems Conference in 2016).

Eerder dit jaar publiceerde Bits&Chips een Engelstalige editie, met daarin mijn artikel Delivering quality software with Agile.

Categories: Blogs

Video: Agile Project Management in 8 minutes

Learn more about transforming people, process and culture with the Real Agility Program

Highlights of this video —

  1. Scrum is not for EVERY project
  2. Scrum is good when there are changing variables & priorities
  3. Scrum is good when there is a cross-functional group of people
  4. Scrum is good when people work together at the same time in the same place (co-located)
  5. The Agile Manifesto isn’t mentioned directly but its alluded to
  6. 4 weeks is the maximum for a Sprint
  7. A “scrum” is a collection of people working together on the project
  8. Backlog items are a list of stories that shows what you want for your end result
  9. Scrum Meetings happen every day!
  10. In a Scrum Meeting you talk about what you did yesterday and what you’ll do today and any obstacles
Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Video: Agile Project Management in 8 minutes appeared first on Agile Advice.

Categories: Blogs

Quotable Quotes: Leverage Technology When Possible!

Learn more about transforming people, process and culture with the Real Agility Program

Jerry Doucett 201601 white background - head - 275 square“For increased chances of success, a Scrum team should leverage technology and engineering practices whenever possible.  Techniques, skills and tools that facilitate Agile approaches such as Continuous Integration, Automated Testing and Test Driven Development all make technical excellence, continuous improvement and truly being “Done” every Sprint a possible reality for a Scrum team.”

(By Senior Agile CoachJerry Doucett)


Jerry is leading a series of SAFe trainings from September to December. More details are here.



Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Quotable Quotes: Leverage Technology When Possible! appeared first on Agile Advice.

Categories: Blogs

Kurt Bittner Joins as Vice President of Enterprise Solutions

Scrum Expert - Tue, 08/23/2016 - 16:53 has announced that Kurt Bittner has joined the company as vice president of enterprise solutions. In this role, Kurt is responsible for enterprise-level professional software delivery initiatives. Kurt Bittner previously worked for Forrester Research, where he served as a Principal Analyst covering how enterprises are adopting Agile and DevOps practices. Prior to Forrester, Kurt served as CTO—Americas for the consulting firm Ivar Jacobson International. “Increasingly, organizations see software as the key for business success, but to do it well, development teams must be professional., which focuses on professional Scrum through assessments and certification, training and licensed Professional Scrum Trainers, is developing enterprise initiatives to provide that organizational environment that enables multiple teams to work together more effectively,” explained Ken Schwaber, co-creator of Scrum and founder of “Kurt brings a wealth of knowledge and experience to this new role, and will help us drive those initiatives for enterprises, which will enable organizations to adapt and thrive in this period of unprecedented challenge and opportunity.” Kurt will work alongside Ken Schwaber and others to evolve existing and create new ways for organizations to build environments that encourage and sustain scaling professional software delivery, and to drive a cohesive vision for on how enterprises can deliver greater value to themselves and their customers by delivering software in a professional way. Bittner joins from
Categories: Communities

My Favorite LeanKit Feature: Subscribe to a Card

Hana here, from Product Marketing at LeanKit. One of my main roles is as a liaison between our Product and...

The post My Favorite LeanKit Feature: Subscribe to a Card appeared first on Blog | LeanKit.

Categories: Companies

Continuous Integration & Delivery at Lego

TV Agile - Tue, 08/23/2016 - 16:36
Security is important – but in my experience managers, product owners and developers often find themselves lost in translation once they attempt to map the “Security is very important to us” abstraction to features and user stories. How do you as a developer protect secrets such as passwords and connection strings without loosing flexibility in […]
Categories: Blogs

Why I Joined LeadingAgile

Leading Agile - Mike Cottmeyer - Tue, 08/23/2016 - 12:30

People who know me well might be surprised to learn I’ve accepted full-time employment. I’m sort of an independent-minded person. So, here’s why. (It’s characteristically long-winded, so skip it if you wish.)


In 2002, I was working as an enterprise architect at a mid-to-large-ish financial services holding company that operated banks, mortgage lenders, investment firms, and so forth. That was my 25th year in the information technology field.

I had reached such a point of frustration with the growing bureaucracy of IT organizations that my wife and I were looking into alternatives, like franchise businesses (Main Event, etc.).

Then a colleague at the company approached me. He said he knew I was on the way out the door, and asked me if I would be willing to work with him on one last thing before I changed careers. He wanted to assess the cost and value of the IT function of the enterprise, and then, based on lessons learned, see what could be done to improve it. I agreed.

Within a few months our small group had learned:
  • The main problem was the existence of functional silos with numerous cross-dependencies. These dependencies caused so much delay that the organization might be likened to a set of interlocking gears so arranged that they couldn’t turn.
  • Every January, the department launched 300 projects at the same time. Staff spent the next seven to eight months waiting to see which ones would crash and burn. Then they scrambled to complete the surviving projects before the December code freeze. January through August was “endless meetings season.” September and October was “death march season.” Then the annual code freeze. Time to recover before the next January.
  • The cost of going through the formal bureaucratic process to request services from the IT department was so high that line-of-business leaders didn’t even bother to request things they wanted. Besides, if they requested something they couldn’t be sure it would ever materialize.
  • Individual contributors had no idea of the context of their work. From their point of view, there was no coherent activity in the IT department; everything was just a series of disconnected requests for small tasks (add an index on this database table; open such-and-such a port on this internal router; add a field to this COBOL copybook; etc.).
  • No one ever received any feedback about the downstream impact of their work. Were the architectural designs sound, or did people have to revamp them? Was the code okay, or did operations have to mess with it to get it installed? Did customers like the user experience, or was it horrid?

In today’s terms, you’ll recognize some of those symptoms as high WIP, dependencies, context-switching, overloaded queues, big-bang delivery, and matrixed organizations.

It was during this effort that we discovered the Agile Manifesto. We recognized in it the same values we were seeking. We reasoned that if there were enough people out in the world who were thinking along the same lines to have published such a document, then someone out there must know how to do this stuff. We looked for help and found ThoughtWorks. We went from there.

I said at the time that if “this stuff” didn’t stick, and the industry reverted to the status quo ante, I would bail. There are other ways to make a living besides the slow death of bureaucracy. I began my Agile journey with one foot out the door. In some ways this has been a Good Thing; it has kept me actively focused on value every day.

Did I bail? No: 2016 is my 39th year in the information technology field.

Here we go again

Ah, but there’s trouble in River City. Today, Agile seems to be all about peddling frameworks. You can become certified to teach a framework certification class by passing a framework certification class. It isn’t necessary to have worked in the IT field at all, let alone to understand how Agile thinking can apply to enterprise IT. There are some 400,000 Certified Scrum Masters, only a few of whom have ever written a line of code, worked as a tester, in operations, or in production support.

Some frameworks are meant for scaling team-level Agile methods. They operate by re-introducing traditional governance methods, and killing agility in the process. (See “Every Agile Scaling Framework in the World“)

Other frameworks are meant for de-scaling the organization. They operate by insisting upon radical structural changes on Day One; changes that are impractical in most cases.

The alternative, Kanban, (as defined by Lean Kanban Inc.) is a business organized around selling training courses. It’s more realistic and practical than any of the Agile frameworks, and yet it stops short of providing concrete guidance to clients. The approach is to train leaders (and coaches) and then let them chart their own path forward. Change their thinking, and then get out of their way. Not a bad approach, actually, but it may not go quite far enough.

Agile conferences are 99% about playing board games, sticking notes on the walls, bending pipe-cleaners, and constructing things with little plastic building blocks; and 1% about addressing business needs through Agile thinking or advancing technical excellence through sound software engineering practices.

Agile consultancies that used to help clients solve problems and achieve goals have devolved into sales organizations for “frameworks.” It’s easy money. I know a number of people involved, and they know better than to do this. It really hurts my heart to see them selling their ethics this way. I guess that sounds maudlin, but at least it’s sincere.

Maudlin or not, I’m far from alone in this assessment of the state of Agile.

Four members of the ScrumAlliance board resigned within two months of the date I wrote this. They gave generic reasons, but two of them mentioned that the organization was going in a direction inconsistent with the values expressed in the Agile Manifesto. (I don’t know how long this link will remain valid, but here’s their letter of resignation.)

More than a few people are disappointed with the way things have been going. Here’s an excerpt from a piece by James Grenning on Quora.

Being one of the people that participated in the creation of the Agile Manifesto, I find myself very disappointed by the reaction of engineers to the question “Are you practicing Agile?” Their shoulders drop. They tell me agile is horrible. I ask why. Reasons that stand out are:

  • We’re being micromanaged
  • The pressure is constantly on for every two week deliveries so quality suffers
  • All they care about is the date

Unfortunately, none of those activities are part of Agile, though I can see how it comes to be. The usual starting point is one of mistrust (note they above). Then you get a Scrum Master with two-days of training and pressure for two week deliveries; engineers will get the idea they are the Scrum Slaves.

Another of the Manifesto’s authors, Ron Jeffries, has had occasion to question the direction in which the ideas have been taken. Here’s an excerpt from an article on his site entitled “What I wish we had done…“.

As a faithful reader, you’re probably aware that I believe that the word “Agile” should mean “consistent with the Agile Manifesto”. And, of course, it doesn’t mean that at all. Instead it means whatever the word’s user means at that moment. There’s great confusion as a result. Ah, well, life is tough, then you die. Quickly, I hope, but not soon. But I digress…

An important contributor to the confusion is the proliferation of branded or named methods and frameworks. At the time of the Manifesto, if I recall, there were Scrum, XP, Crystal, and DSDM. Anyway right around that time, Jim Highsmith came out with Adaptive Software Development, Mike Beedle began a series of names like XP@Scrum, and it went on and on. It continues to this day.

…Point is, there are a lot of them. I don’t think we really need a “method” at all. I feel quite sure we don’t need a dozen…

Dave Thomas wrote, in a 2014 piece entitled, “Time to Kill Agile”.

The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products.

Another Manifesto author, Brian Marick, has recently reanimated his 2009 idea, “Artisanal Retro-Futurism Crossed With Team-Level Anarcho-Syndicalism.” The name is designed to be descriptive of the underlying concept while simultaneously hard for people to cargo-cult. It’s intentionally non-catchy. The catchiness of the word “agile” has backfired.

Those comments come from authors of the Agile Manifesto. They aren’t alone. There’s a lot of disillusionment going around these days. I sympathize. Sometimes it feels as if the entire Agile movement is going in a direction inconsistent with the values expressed in the Agile Manifesto.

The whole thing with “scaling frameworks” has gotten totally out of hand. On a recent subcontracting engagement, I was tasked with helping the prime contractor’s client adopt the Scaled Agile Framework, a.k.a. SAFe. It didn’t take long to observe that the framework was a poor fit for the organization’s transformation goals. They were wasting massive amounts of time, effort, and money on it. But I couldn’t say so, as the prime contractor was pushing SAFe to the exclusion of any other ideas or approaches; and they were only pushing the outward appearance of compliance with SAFe’s “rules.” They were doing nothing of substance. I didn’t last long on that engagement, and that’s okay with me. It made me feel soiled.

That’s not an isolated example, or an aberration. It’s the norm when it comes to companies trying to adopt one “scaling framework” or another. It’s all about the “rules.” There’s no attempt to learn to think in a “lean” or “agile” way. And middleman companies are all too eager to stand under the rain of cash, arms wide and mouths agape, soaking up the profits. People who know better are hawking frameworks like stereotypical used car salesmen. Ethics, anyone?

By January of 2016, I had one foot out the door…again. The reason for the long gap since my last contract is that I’ve been taking a real estate training course, to become a licensed real estate agent. “This stuff” didn’t stick. Time to leave.

Enter LeadingAgile

Then a colleague in the Agile community approached me. He said, hey, before you leave, take a look at this. He pointed me to his company’s website:

Within a couple of months I learned:

  • There are still people in the Agile community who are genuinely interested in helping clients solve problems and achieve goals.
  • There are people in the Agile community who understand an established company can’t abruptly restructure itself and “go agile.”
  • There are people in the Agile community who recognize the value of a framework when it adds value, but who do not “push” a framework.
  • There are people in the Agile community who have a realistic model of organizational transformation, a way to craft a practical roadmap tailored to a client’s situation, and the willingness to guide the client along a path to improvement over the long term.

Many of those people work at LeadingAgile.

LeadingAgile has a very pragmatic and results-focused approach to organizational transformation. Some people think the word pragmatic implies backing off from Agile principles; actually, it means something like practical and realistic. Moving an organization toward business agility means defining a rational system of delivery built around teams. It means paying attention to flow, by which we mean the flow of value, not just “busyness.” Ideas from Lean Thinking are key to the approach, including high visibility and paying close attention to delivery rate and, by implication, anything that impedes value flow or delivery. This practical-mindedness has great appeal for me.

I discovered several people I knew were working at LeadingAgile. Some I had worked with in the past. Some I knew through Agile community events. Some I knew through their work. Mike Cottmeyer, Dennis Stevens, Chris Beale, Tom Churchwell, Rachel Howard, Derek Huether. A who’s who. An all-star team.

For me, then, joining LeadingAgile is like coming home. Home to people I know, whose views about Agile are aligned with my own, and whom I respect professionally and personally. Home to the core values of Agile, which have become so watered down and corrupt over the years.

That’s why.
Now, if “this stuff” doesn’t stick…

The post Why I Joined LeadingAgile appeared first on LeadingAgile.

Categories: Blogs

Agenda and Ideas

BigVisible Solutions :: An Agile Company - Tue, 08/23/2016 - 10:00

Retrospectives can easily devolve into chaotic gripe sessions or dull affairs with low participation unless they are well-planned and executed. At each extreme, the meeting suffers from poor facilitation, which hurts morale and makes it difficult to generate concrete action items. To steer the meeting toward successful outcomes, the Scrum Master has a full set of meeting facilitation tools at his or her disposal.

One such tool is the Retrospective agenda, which is a time-boxed outline that serves as the template for the meeting’s activities. It ensures that the Retrospective progresses in a logical, step-wise fashion. With a thoughtfully crafted agenda in hand, the Scrum Master can lead the team from behind to its own fresh and actionable insights, while ensuring it doesn’t go off course into unproductive territory.

Below, we’ll look at various activity types that comprise an effective agenda. Each generates specific outcomes; in combination, they lead to agreement on actionable improvements. Within each type, you can select from an array of popular and tested activities. This allows you to keep your meetings fresh and get your participants to think outside the box. Note that your agenda is filled with “activities” rather than “discussions.” Activities can and do lead to discussions, but they can generate more personal and unexpected insights than asking the three Retrospective questions directly, sprint after sprint.

The following is a recommended agenda for a 1 hour Retrospective. There are different formulas out there for retrospective time per weeks in the sprint. 1 hour to 1-1/2 hours for a 2-week sprint should serve quite well. If you engage your soft skills of scanning the room and leading from behind, you’ll be able to judge what’s best for your team. If the activities are tightly run without rambling discussions, 1 hour should work fine for a standard team of 6-9 members. You’ll get better results if you keep things fairly fast paced, so lean towards a shorter versus a longer meeting and keep things moving. Your team will thank you for getting to a quick conclusion.

Here’s a recommended agenda for a 1-hr retrospective!
Click To Tweet

Icebreaker (10 minutes)

Getting people into the same room doesn’t constitute a meeting if you’ve co-located their bodies without co-locating their minds. As stated earlier, you are essentially architecting a network of complex hardware (brains) and software (personalities). Check-in activities boot that system up. Minds tend to wander, especially among high performing individuals who obsessively multitask. Even though electronics may be put away, you can be sure someone is working out a line of code or writing an e-mail or grocery list in their head. Your check-in activity will break these pre-meeting thought strings and get all those minds co-located and ready to collaborate.

Another function of check-ins is to establish an upbeat, friendly, and dare I say, FUN, atmosphere. We want to shake off any self-consciousness, hesitation, or insecurity. We want to open up the flood gates of free, honest, and creative contributions. Seriousness puts a damper on participation, as people tend to censor themselves and share only the most carefully considered, “right” answers. When it comes to brainstorming, we want all ideas on the table without prejudice. This is easiest when people are in a somewhat playful, unrestrained mood. So we have a two-fold purpose for our check-in: 1) clear away distracting concerns; 2) set an upbeat and collaborative tone.

How to break the ice in your next retro? Here are three tips for you.
Click To Tweet

Here are just a few activities to choose from:

Mind Dump

A very direct way to defuse external distractions is the “Mind Dump.” Each participant has two large sticky notes. On one, they write what they were doing before the meeting; on the other, they write what they’ll be doing immediately after the meeting. They post these on a white board or wall under the labels “Before” and “After.” The Scrum Master reads each list, asking, “Is there anything here that can’t wait?” or “Is there something here that you need help with?” Any such wording will work. The goal is to identify and quell any potential distractions.

Identification Exercises

These allow people to reveal something unique about themselves to each other. Some options include: “Which super hero or movie character would you like to be, and why?” “If we could hold this meeting in the place of your choice, where would we be, and why?” “What’s the best thing that happened last weekend?” You can ask any open-ended question which reveals personal preferences. The goal is to help team members relate to each other and set the stage for uninhibited collaboration.

Improv Exercises

Various improvisation exercises can be used to get people interacting. In “What’s in my Box?” team members pair up, with one partner holding an imaginary box. The other partner mimes removing items from the box and announces what they are. “Here’s a nice chocolate cake.” To which the partner replies, “Yes, that’s a nice chocolate cake.” This continues for two minutes and then the partners reverse roles. Yes, this is silly! That’s the point. In the Retrospective they may be confronting sensitive issues. The purpose of this check-in is to encourage freedom of expression and empathetic listening. Participants will sometimes pull strange items like airplanes, planets, or dinosaurs from their boxes, but the only acceptable response is, “That’s a nice ________.”

Check-ins may be light-hearted, but they’re not lightweight. As long as humans are involved, there will be underlying inhibitions and distractions that impede free and productive brainstorming sessions. Once those are addressed, people will lower their guards and contribute more openly and honestly. The check-in sets the stage for the upcoming more serious swork.

Data Review (5 minutes)

In the Retrospective, you want to elicit personal reflections on how the sprint went and get unique insights on what can be improved. First, you must be sure everyone is looking at the same sprint. Presenting hard sprint data for group review narrows their focus to the facts at hand. Clear and easily digested charts should be visible to all in the room. A dashboard view of the sprint should include your burndown, defect counts, stories completed, stories deferred, outcomes of any critical meetings, sprint impacting events, etc. The raw data is presented objectively and free from any interpretation by the Scrum Master, the Product Owner, or any other team member. There are no “good” or “bad” facts here; there are only facts


Have the team look, but don’t allow any discussions just yet. You want them to absorb the information, process it individually, and stir up their own personal opinions about what to do better in the next sprint. Because group discussions can get quite unwieldy and unproductive, Scrum best practices tend to favor silent brainstorming, using a highly sophisticated information management tool – the Post-it note!

Eliciting (10 minutes)

Once your team has been allowed to reflect on the simple, mathematical facts of the sprint, you will want to elicit their feedback as to what went well, what didn’t go so well, and what could be done better. The most straightforward way to do this is to hand each member a small pad of 3X5 inch Post-it notes and a fine-tipped marker. Have them answer each question as many times as they wish, putting each brief response on a separate note. They will stick them to a prepared white board or wall with a column set up for each question: What did we do well? What didn’t we do so well? What could we do better?

In time, you may want to vary the elicitation activity to keep things interesting and encourage fresh perspectives. For example, the “Speedboat Retrospective” uses the metaphor of a speedboat for the sprint and elicits replies to four boating-inspired questions: 1) What propels us forward? (represented by an outboard motor); 2) What slows us down? (represented by an anchor); 3) Where do we crash? (represented by rocks); and 4) What rescues us? (represented by a life preserver). The scrum master sets these four quadrants up on a wall or white board, with images of the four icons (motor, anchor, rocks, and life preserver) as headers. Participants attach their notes in the appropriate quadrants.

Another fairly flexible template I created is what I call the “Wheel of Fortune.” I draw a large, six-sectioned pie chart with 3 pairs of opposite categories, for example: fast/slow, easy/hard, clear/unclear; or won/lost, coordinated/uncoordinated, continue/stop. Partner categories are located opposite each other to keep the concepts clear and simplify posting. The key is to keep the categories as general as possible, so as not to limit potential responses.

Varying your elicitation exercise will keep the team on their toes and lead them towards less obvious problems and solutions. They may come into the meeting armed with pre-conceived answers to the three basic questions, but if you stir things up with unusual or unexpected questions you’re likely to generate some fresh ideas. The key here is to elicit a wide range of responses. They will be organized into manageable groups after they surface.

Grouping (10 minutes)

Once you’ve captured these impressions, you can start grouping identical, similar, or related responses. Identical, or very similar, responses can be placed over each other; related responses can be placed near each other, and unrelated responses should be kept apart. This first phase of grouping can be done as the scrum master reads out the responses in a particular section, identifies similarities, and asks the team if particular items should be grouped together. Alternatively, the team can gather around the board and group items themselves, with minimal discussion for clarification and consensus only. When replies are grouped, the Scrum Master or another team member should write a summary statement on a new Post-it and place it over the group. Once the notes are grouped they can be sorted in the next activity.

Sorting (10 minutes)

Once you’ve grouped your Post-its to eliminate redundancies and show dependencies, you can sort them into action oriented categories to identify potential action items. A fairly simple and robust framework is Stop, Continue, and Start. Work with the team to identify which notes represent practices which should be stopped, continued, or started, or let them sort amongst themselves. For example, un-productive activities would be placed under a “Stop” heading. Items that call for new and better ways to do things would be placed under “Start.” Successful actions would be placed under “Continue” to ensure they’re not forgotten. Moving notes from the previous activities into these three categories is a precursor to creating concrete action items with assigned owners. Not all of them could or should be moved. The idea of this sorting activity is to deliberately slow down the thinking process. The team isn’t deciding what to do yet; it’s only considering what it might do (stop, start, or continue something) to address an issue. In the final activity, the team will choose action items from this pre-sorted list.

Action Items (10-15 minutes)

The team has already narrowed down its improvement candidates with the Stop-Start-Continue list, now they can decide which items to commit to. This is an activity where group discussion is most likely to be manageable and productive, since the scope of that discussion is fairly limited.  As consensus builds around particular items, they are moved to an “Action Items” list, but that shouldn’t happen unless someone steps up as the “owner.” The owner drives and monitors progress. He or she may contribute directly, may coordinate the contributions of others, or may do both, but regardless, that person will spearhead the effort and drive it to conclusion. This is an area where volunteerism is your best friend. Unless someone is motivated to drive results, an action item is useless.  So, it is okay if “more important” items aren’t taken up because no one steps up to own them. Without owners, they wouldn’t be completed anyway. It’s far better to complete lesser initiatives than to do nothing about the more important ones. The only criteria that matter are group consensus and individual ownership.

It’s the Scrum Master’s responsibility to record and disseminate these improvement items so that they’re not forgotten, remove impediments and ensure that progress is reported to the team. Some teams place these improvement items in their sprint backlog to give high visibility. The daily scrum is an opportunity to report progress; the team can also utilize team discussion pages, posters, email/chat communication, etc., to get the word out on any behavioral changes needed to support the action items.


In the Retrospective, we’re not just dealing with simple facts and computations; we’re dealing with human beings in all their complexity. We are consulting them individually and collectively to elicit their insights on what they might do better. There are no obviously right or wrong answers, there are only recommendations that are implemented and tested.

It takes good planning and execution to have a successful Retrospective.  An agenda of activities safeguards the team from distractions, diversions, and divisions.  As the team moves through a logical progression of activities, they are led to individual insights and group consensus on improvement opportunities.

A successful Retro takes good planning and execution.
Click To Tweet


The Agile mantra, “Inspect and Adapt” is realized in the Sprint Retrospective. High performing teams are always reaching for the next level of excellence. The Retrospective exposes strengths, weaknesses, and opportunities for improvement. Some call these “improvement experiments” or “hypotheses,” since there are no guarantees they’ll have the desired impact. That may be so, but as long as intelligent and deliberate effort is made, and as long as the team continues to course correct and explore new opportunities with each Retrospective, overall improvement is inevitable as successful new practices are adopted and detrimental practices are eliminated.


Looking for a visualization of this awesome retrospective agenda and idea? Look no further.

60-min Retro_Cover-01

Download Now

The post Agenda and Ideas appeared first on SolutionsIQ.

Categories: Companies

Alignment at Scale – slides from my Agile Africa keynote

Henrik Kniberg's blog - Tue, 08/23/2016 - 09:35

Here are the slides from my Agile Africa keynote Alignment at Scale (or How to Not become Totally Unagile when you have Lots of Teams). Thanks for a great conference!

And thanks everyone for the Emma greeting, that sure made an 8 year girl very happy :)

(Emma was supposed to join me on this trip, but couldn’t make it because I had missed some required paperwork for travelling with minors to South Africa).

Agile Alignment at Scale

Too busy to improve

Agile leadership


Categories: Blogs

AgileByExample, Warsaw, Poland, October 10-12 2016

Scrum Expert - Tue, 08/23/2016 - 09:00
AgileByExample is a Lean and Agile three-days conference taking place in Warsaw, Poland that helps you learn Agile on live examples. The first day is dedicated to a Lean Agile Dojo with workshops. Keynotes, talks and discussions are all in English. In the agenda of the AgileByExample conference you can find topics like “5 Whys Root Cause Analysis”, “Taking Back Agile: Returning to our Core Values and Practices”, “No time for Planning Poker? Try Silent Sort Estimating Instead”, “eXPert Leadership”, “Time to Clean up the Product!”, “Pair Programming Demystified”, “The Evolution of a Super Agile Software Development Capability”, “Scaling agile at LEGO”, “Choosing Change: How to Enable a Shift to Agile”. Web site: Location for the AgileByExample conference: Multikino Ursynów, Al. KEN 60, Imielin Metro Station, Warsaw, Poland
Categories: Communities

Neo4j/scikit-learn: Calculating the cosine similarity of Game of Thrones episodes

Mark Needham - Mon, 08/22/2016 - 23:12

A couple of months ago Praveena and I created a Game of Thrones dataset to use in a workshop and I thought it’d be fun to run it through some machine learning algorithms and hopefully find some interesting insights.

The dataset is available as CSV files but for this analysis I’m assuming that it’s already been imported into neo4j. If you want to import the data you can run the tutorial by typing the following into the query bar of the neo4j browser:


Since we don’t have any training data we’ll be using unsupervised learning methods, and we’ll start simple by calculating the similarity of episodes based character appearances. We’ll be using scitkit-learn‘s cosine similarity function to determine episode similarity.

Christian Perone has an excellent blog post explaining how to use cosine similarity on text documents which is well worth a read. We’ll be using a similar approach here, but instead of building a TF/IDF vector for each document we’re going to create a vector indicating whether a character appeared in an episode or not.

e.g. imagine that we have 3 characters – A, B, and C – and 2 episodes. A and B appear in the first episode and B and C appear in the second episode. We would represent that with the following vectors:

Episode 1 = [1, 1, 0]
Episode 2 = [0, 1, 1]

We could then calculate the cosine similarity between these two episodes like this:

>>> from sklearn.metrics.pairwise import cosine_similarity
>>> one = [1,1,0]
>>> two = [0,1,1]
>>> cosine_similarity([one, two])
array([[ 1. ,  0.5],
       [ 0.5,  1. ]])

So this is telling us that Episode 1 is 100% similar to Episode 1, Episode 2 is 100% similar to itself as well, and Episodes 1 and 2 are 50% similar to each other based on the fact that they both have an appearance of Character B.

Note that the character names aren’t even mentioned at all, they are implicitly a position in the array. This means that when we use our real dataset we need to ensure that the characters are in the same order for each episode, otherwise the calculation will be meaningless!

In neo4j land we have an APPEARED_IN relationship between a character and each episode that they appeared in. We can therefore write the following code using the Python driver to get all pairs of episodes and characters:

from neo4j.v1 import GraphDatabase, basic_auth
driver = GraphDatabase.driver("bolt://localhost", auth=basic_auth("neo4j", "neo"))
session = driver.session()
rows ="""
    MATCH (c:Character), (e:Episode)
    OPTIONAL MATCH (c)-[appearance:APPEARED_IN]->(e)
    RETURN e, c, appearance
    ORDER BY,""")

We can iterate through the rows to see what the output looks like:

>>> for row in rows:
        print row
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5415 labels=set([u'Character']) properties={u'name': u'Addam Marbrand', u'id': u'/wiki/Addam_Marbrand'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5882 labels=set([u'Character']) properties={u'name': u'Adrack Humble', u'id': u'/wiki/Adrack_Humble'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=6747 labels=set([u'Character']) properties={u'name': u'Aegon V Targaryen', u'id': u'/wiki/Aegon_V_Targaryen'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5750 labels=set([u'Character']) properties={u'name': u'Aemon', u'id': u'/wiki/Aemon'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5928 labels=set([u'Character']) properties={u'name': u'Aeron Greyjoy', u'id': u'/wiki/Aeron_Greyjoy'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5503 labels=set([u'Character']) properties={u'name': u'Aerys II Targaryen', u'id': u'/wiki/Aerys_II_Targaryen'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=6753 labels=set([u'Character']) properties={u'name': u'Alannys Greyjoy', u'id': u'/wiki/Alannys_Greyjoy'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=6750 labels=set([u'Character']) properties={u'name': u'Alerie Tyrell', u'id': u'/wiki/Alerie_Tyrell'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5753 labels=set([u'Character']) properties={u'name': u'Alliser Thorne', u'id': u'/wiki/Alliser_Thorne'}> appearance=None>
<Record e=<Node id=6780 labels=set([u'Episode']) properties={u'season': 1, u'number': 1, u'id': 1, u'title': u'Winter Is Coming'}> c=<Node id=5858 labels=set([u'Character']) properties={u'name': u'Alton Lannister', u'id': u'/wiki/Alton_Lannister'}> appearance=None>

Next we’ll build a ‘matrix’ of episodes/characters. If a character appears in an episode then we’ll put a ‘1’ in the matrix, if not we’ll put a ‘0’:

episodes = {}
for row in rows:
    if episodes.get(row["e"]["id"]) is None:
        if row["appearance"] is None:
            episodes[row["e"]["id"]] = [0]
            episodes[row["e"]["id"]] = [1]
        if row["appearance"] is None:

Here’s an example of one entry in the matrix:

>>> len(episodes)
>>> len(episodes[1])
>>> episodes[1]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

From this output we learn that there are 60 episodes and 638 characters in Game of Thrones so far. We can also see which characters appeared in the first episode, although it’s a bit tricky to work out which index in the array corresponds to each character.

The next thing we’re going to do is calculate the cosine similarity between episodes. Let’s start by seeing how similar the first episode is to all the others:

>>> all = episodes.values()
>>> cosine_similarity(all[0:1], all)[0]
array([ 1.        ,  0.69637306,  0.48196269,  0.54671752,  0.48196269,
        0.44733753,  0.31707317,  0.42340087,  0.34989921,  0.43314808,
        0.36597766,  0.18421252,  0.30961158,  0.2328101 ,  0.30616181,
        0.41905818,  0.36842504,  0.35338088,  0.18376917,  0.3569686 ,
        0.2328101 ,  0.34539847,  0.25043516,  0.31707317,  0.25329221,
        0.33342786,  0.34921515,  0.2174909 ,  0.2533473 ,  0.28429311,
        0.23026565,  0.22310537,  0.22365301,  0.23816275,  0.28242289,
        0.16070148,  0.24847093,  0.21434648,  0.03582872,  0.21189672,
        0.15460414,  0.17161693,  0.15460414,  0.17494961,  0.1234662 ,
        0.21426863,  0.21434648,  0.18748505,  0.15308091,  0.20161946,
        0.19877675,  0.30920827,  0.21058466,  0.19127301,  0.24607943,
        0.18033393,  0.17734311,  0.16296707,  0.18740851,  0.23995201])

The first entry in the array indicates that episode 1 is 100% similar to episode 1 which is a good start. It’s 69% similar to episode 2 and 48% similar to episode 3. We can sort that array to work out which episodes it’s most similar to:

>>> for idx, score in sorted(enumerate(cosine_similarity(all[0:1], all)[0]), key = lambda x: x[1], reverse = True)[:5]:
        print idx, score
0 1.0
1 0.696373059207
3 0.546717521051
2 0.481962692712
4 0.481962692712

Or we can see how similar the last episode of season 6 is compared to the others:

>>> for idx, score in sorted(enumerate(cosine_similarity(all[59:60], all)[0]), key = lambda x: x[1], reverse = True)[:5]:
        print idx, score
59 1.0
52 0.500670191678
46 0.449085146211
43 0.448218732478
49 0.446296233312

I found it a bit painful exploring similarities like this so I decided to write them into neo4j instead and then write a query to find the most similar episodes. The following query creates a SIMILAR_TO relationship between episodes and sets a score property on that relationship:

>>> episode_mapping = {}
>>> for idx, episode_id in enumerate(episodes):
        episode_mapping[idx] = episode_id
>>> for idx, episode_id in enumerate(episodes):
        similarity_matrix = cosine_similarity(all[idx:idx+1], all)[0]
        for other_idx, similarity_score in enumerate(similarity_matrix):
            other_episode_id = episode_mapping[other_idx]
            print episode_id, other_episode_id, similarity_score
            if episode_id != other_episode_id:
                    MATCH (episode1:Episode {id: {episode1}}), (episode2:Episode {id: {episode2}})
                    MERGE (episode1)-[similarity:SIMILAR_TO]-(episode2)
                    ON CREATE SET similarity.score = {similarityScore}
                    """, {'episode1': episode_id, 'episode2': other_episode_id, 'similarityScore': similarity_score})

The episode_mapping dictionary is needed to map from episode ids to indices e.g. episode 1 is at index 0.

If we want to find the most similar pair of episodes in Game of Thrones we can execute the following query:

MATCH (episode1:Episode)-[similarity:SIMILAR_TO]-(episode2:Episode)
WHERE ID(episode1) > ID(episode2)
RETURN "S" + episode1.season + "E" + episode1.number AS ep1, 
       "S" + episode2.season + "E" + episode2.number AS ep2, 
       similarity.score AS score
ORDER BY similarity.score DESC
│ep1  │ep2 │score             │
│S1E2 │S1E1│0.6963730592072543│
│S1E4 │S1E3│0.6914173051223086│
│S1E9 │S1E8│0.6869464497590777│
│S3E7 │S3E6│0.6819943394704735│
│S2E7 │S2E6│0.6813598225089799│
│S1E5 │S1E4│0.6698105143372364│
│S4E5 │S4E4│0.6518358737330705│

And the least popular?

MATCH (episode1:Episode)-[similarity:SIMILAR_TO]-(episode2:Episode)
WHERE ID(episode1) > ID(episode2)
RETURN "S" + episode1.season + "E" + episode1.number AS ep1, 
       "S" + episode2.season + "E" + episode2.number AS ep2, 
       similarity.score AS score
ORDER BY similarity.score
│ep1 │ep2 │score              │
│S4E9│S1E5│0                  │
│S4E9│S1E6│0                  │
│S4E9│S4E2│0                  │
│S4E9│S2E9│0                  │
│S4E9│S2E4│0                  │
│S5E6│S4E9│0                  │
│S6E8│S4E9│0                  │
│S4E9│S4E6│0                  │

The output of this query suggests that there are no common characters between 8 pairs of episodes which at first glance sounds surprising. Let’s write a query to check that finding:

MATCH (episode1:Episode)<-[:APPEARED_IN]-(character)-[:APPEARED_IN]->(episode2:Episode)
WHERE episode1.season = 4 AND episode1.number = 9 AND episode2.season = 1 AND episode2.number = 5
return episode1, episode2
(no changes, no rows)

It’s possible I made a mistake with the scraping of the data but from a quick look over the Wiki page I don’t think I have. I found it interesting that Season 4 Episode 9 shows up on 9 of the top 10 least similar pairs of episodes.

Next I’m going to cluster the episodes based on character appearances, but this post is long enough already so that’ll have to wait for another post another day.

Categories: Blogs

Announcements, Links & Entertainment

Learn more about transforming people, process and culture with the Real Agility Program

For 10 years, Agile Advice has been providing useful information, Agile-related announcements & entertaining content for Agile ambassadors.

But Agile Advice is just one of the many portals to the BERTEIG online network.

For your easy reference here is a list of other 5 links which may bring you to the information you are looking for.
  1. Register for Training & Learning Events 
  2. Sign Up for BERTEIG’s Loyalty Program
  3. Find out More About BERTEIG’s Corporate Experience
  4. Join the 1000+ others in the LinkedIn Worldmindware Group
  5. Visit the BERTEIG-hosted Facebook Scrum Group with 2600+ members
Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Announcements, Links & Entertainment appeared first on Agile Advice.

Categories: Blogs

Knowledge Sharing

SpiraTeam is a agile application lifecycle management (ALM) system designed specifically for methodologies such as scrum, XP and Kanban.