Skip to content

Feed aggregator

Using Customer Journey Maps

Scrum Expert - Mon, 08/29/2016 - 16:20
If user stories are the start of the conversations to define user requirements, Scrum teams can also use other tools to obtain a more precise definition of these requirements. In the article “When and How to Create Customer Journey Maps”, Kate Williamson presents the concept of customer journey map, the visualization of the process that a person goes through in order to accomplish a goal, and when and how to use them. The article starts with a definition of customer journey maps. To help teams understand and address customer requirements, the journey maps combine two instruments: storytelling and visualization. Journey maps take different forms depending on context and business goals, but they usually start by compiling a series of user goals and actions into a timeline. User thoughts and emotions are added to this timeline to create a narrative. Finally, this narrative is condensed into a visualization used to communicate insights that will inform design processes. After discussing the reason to use customer journey maps, Kate Williamson presents their key elements: * Point of view: choose the actor of the story. * Scenario: determine the specific experience to map. * Actions, mindsets and emotions: what the user is doing, thinking, and feeling during the journey. * Touchpoints and channels: the map aligns touchpoints (times when the actor in the map actually interacts with the company) and channels (methods of communication or service delivery, such as the website or physical store) with user goals and actions. * Insights and ownership: uncover [...]
Categories: Communities

How to Get Started with Continuous Improvement

Not sure how to get started with continuous improvement? Understanding how to use data to drive improvement efforts...

The post How to Get Started with Continuous Improvement appeared first on Blog | LeanKit.

Categories: Companies

So, You’re Saying Docker Isn’t A Virtual Machine???

Derick Bailey - new ThoughtStream - Mon, 08/29/2016 - 13:30

The next time you’re at an art show or art gallery of any kind, walk up to an artist and ask them to install an electrical outlet in the wall on which their painting is displayed. 

After ignoring the inevitable look of bewilderment and snarky response, grab some tools and start hacking the wall apart. Tell the artist that you just need an outlet here so you can recharge your phone.

Be sure to ask them to help.

Artist destroying wall

I’m not 100% certain how this will end, but I can imagine it will involve security guards, police and possibly handcuffs.

As ridiculous as this scenario sounds, though, I feel like this is what I’ve been doing with Docker for the last few months. 

I wanted to access my app via the container’s IP address.

When using the default “bridge” network, a docker container has it’s own IP address. This can be viewed via the “docker inspect <container name>” command.

That being the case, it would make sense to access the app via that IP address, right?

I thought it made sense, and I expected to be able to use the containers IP address.

So, I dug into the documentation.

I googled.

I asked on twitter and in various slack communities.

Finally, I asked on the Docker forums. And the response I got?

Basically, a big “nope” and “don’t do that.”


But I wasn’t happy with that answer.

I continued to suggest that it should work this way, even though it doesn’t.

Sure, it may be possible on linux … or with kubernetes… or some other strange mechanism that I’m not aware of, yet.

It took a while to sink in. Eventually, I realized what the real problem was, though.

I shouldn’t even be trying to do this.

After a few hours of … “conversation” … with my friend Fred, via the WatchMeCode community slack, I finally started to understand that my expectations of Docker were wrong.

I use a lot of of virtual machines to host services that I don’t want on my laptop directly. Things like Oracle, SQL Server, an LDAP server, etc. 

Virtual machines are great at handling these.

Lately, though, I’ve been moving some of these services to Docker. RabbitMQ, MongoDB and my Oracle XE installation are all in Docker containers, for example. 

Life is great when I publish the needed ports from the container through my localhost. Everything works peachy.

My experience with virtual machines clouded my Docker expectations, though.

I saw that Docker was technically running on top of a virtual machine, and I immediately went into a virtual machine mentality and expectation set.

But here’s the thing…

A Docker container is not a virtual machine.

Yes, it may technically be running on a virtual machine, but that is an implementation detail – hidden behind the scenes – not a feature to be used.

An application hosted in a Docker container is a “virtualized” (containerized) application, not a virtual machine. 

Yes, you can find Docker images that include a full linux distribution in them.

These distributions are there to support the application that is contained within, however. They are not to be executed as if they were a full, general purpose virtual machine.

This perspective matters, because it sets the expectations for a containerized app.

Imagine for a moment, that you have two node / express apps that both use port 3333.

What happens when you try to run both of them on the same machine?


You get a big fat error, saying the port is already in use.

Now, does it matter that the application is hosted in a Docker container?

Not really – at least, not from the perspective of the host machine when you try to bind port 3333 to more than one container.

You can’t run two apps on the same port of the same IP address. Not with any app on your machine directly, and not with Docker.

And why, you might ask? Because …

A Docker container is application virtualization.

And application virtualization is not the same as building a complete virtual machine to run a single service.

A Docker container happens to contain everything that the application needs, to run. But the intent and behavior of that container is to run the application on behalf of your host machine. 

It’s a black box that allows you to cleanly separate applications that need different versions of the same dependency.

It gives you the ability to stop and start a server or service as if it were running on your computer directly, without having it installed or configured on your computer.

Docker’s job is to let you sandbox different apps in a system, so they don’t clobber each other’s configuration.

A Docker container is art hanging on a wall.

Art occupies a physical space. And due to those pesky laws of physics, one piece of art will prevent another from occupying the same space.

This was the mistake I made in my view of Docker.

I expected each Docker container to be a separate art gallery. In reality, each container is a piece of art hanging on the wall.

I should have been asking the artist about the intent or meaning of their painting, enjoying it for what it is.

Instead, I wanted to know why there wasn’t an outlet in the wall. I wanted the artist to help me with construction work – something that they may know nothing about.

I decided to try and tear up the wall on which the art was mounted, because I thought the wall was a fundamental part of the installation.

And the installation, as I saw it, wasn’t living up to my very inaccurate expectations.

Categories: Blogs

Manage Agile, Berlin, Germany, November 15-18 2016

Scrum Expert - Mon, 08/29/2016 - 09:00
The Manage Agile conference is a four-day event taking place in Berlin that focused on Agile project management approaches. It is divided into two workshop days and two conference days. The conferences focuses on management topics and is a networking platform where specialists and managers compare notes yearly to establish Agile topics not only in software engineering but also in the whole company up to the management. Most of the talks are in German but there are also a lot of talks in English. In the agenda of the Manage Agile conference you can find topics like “Agile Estimating and Planning”, “SAFe City Simulation – The Scaled Agile Framework in Action!”, “Empowering teams to take ownership of their continuous improvement”, “The Predictability of Delivery”, “The-Poke-Concept – The key to value based development”, “More with LeSS: A Decade of Descaling with Large-Scale Scrum”, “90 Day Challenge – Agile on a Corporate Scale”, “Our community is our customer – diversity challenges agile”, “Ten strategies to (mis)manage Agile”, “Estimation Is Waste!”, “What’s the Value of Agile? A Look at Managing Finances in Agile Projects”. Web site: Location for the Manage Agile conference: RAMADA Hotel Berlin Alexanderplatz, Karl-Liebknecht-Strasse 32, 10178 Berlin, Germany
Categories: Communities

Quotable Quote: Consistent Commitment is Key To Team Success

Learn more about transforming people, process and culture with the Real Agility Program

Jerry Doucett 201601 white background - head - 275 square

There must be a consistent commitment and engagement from all parties in the organization towards adopting the Scrum framework, Agile methods, and thinking.  The initiative must be an open, collaborative experience and there must be complete understanding  and alignment by all parties in assuming the risks and rewards as well as sharing in the effort.  This includes not only business partners and their IT counterparts, but their leadership as well as all of the people and teams supporting an Agile initiative. (Senior Agile Coach Jerry Doucett)

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Quotable Quote: Consistent Commitment is Key To Team Success appeared first on Agile Advice.

Categories: Blogs

Reviewing the latest blinks August 28 - Sun, 08/28/2016 - 17:50

Brain Rules: 12 Principles for Surviving and Thriving at Work, Home and School by John Medina – A description of rules with how our brain works and how we learn. Our visual senses tend to trump our sense of smell. We need sleep to restore our energy and to help us concentrate. Spaced repetition is important, but assigning meaning to new words and concepts are also important to learning. Since I’m fascinated with learning and how the brain works, I’ll add this to my reading list.

Getting Things Done: The Art of Stress-free Productivity by
David Allen
– Although I never read the book, I felt like I follow a similarly described organisation system. The GTD method is almost like a cult, but requires a lot of discipline for it. Unlike keeping a single list of things to do, they have a systemised variant for keeping long-lived projects and ways of managing tasks to help you focus on getting through actions. Probably a good book if you want to focus more on breaking things done into smaller steps.

The Checklist Manifesto: How to Get Things Right by Atul Gawande – With lots of examples from the healthcare industry, a reminder that useful checklists can help us avoid making simple mistakes. For me, the idea of standardised work (a lean concept) already covers this. I agree with this idea in principle, but I’m not so sure the book covers the negative side effects of checklists as well (people getting lazy) or alternatives to checklist (automation and designing against error/failure demand to be begin with).

Connect: The Secret LinkedIn Playbook to Generate Leads, Build Relationships, and Dramatically Increase Your Sales by Josh Turner – Either a terrible summary or a terrible book, this blink gave advice about how to use LinkedIn to build a community. Although the advice isn’t terrible, it’s not terribly new, and I didn’t really find any insights. I definitely won’t be getting a copy of this book.

Start With Why: How Great Leaders Inspire Everyone To Take Action by Simon Sinek – A nice summary of leadership styles and rather than focusing on how something should be done, and the what, is starting with the why. I liked the explanation of the Golden Circle with three concentric circles draw within each other, with the Why being the starting point that leads to the How that ends in the What. It’s a good reminder about effective delegation and how powerful the Why motivator can be. I’ve added this book to my reading list to.

Categories: Blogs

Video Link: Understanding Cloud Concepts

Learn more about transforming people, process and culture with the Real Agility Program

This is a great introduction to Cloud Concepts!

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Video Link: Understanding Cloud Concepts appeared first on Agile Advice.

Categories: Blogs

Announcement: New Leadership Training – First in Canada!

Learn more about transforming people, process and culture with the Real Agility Program Certified Agile Leadership (CAL 1) Training Michael Sahota - Profile Picture (2016)Introduction:

Advanced training for leaders, executives and change agents working in Agile environments.

Your success as a leader in an Agile organization requires looking beyond Agile itself. It requires a deep understanding of your organization and your own leadership path. To equip you for this journey, you will gain a strong foundation in understanding organizational culture. From there, you will learn key organization and leadership models that will allow you to understand how your organizational culture really works.

Now you are ready to start the journey! You will learn about organizational growth – how you may foster lasting change in your organization. Key is understanding how it invite change in a complex system. You will also learn about leadership – how you may show up more effectively. And how to help others.

Learning Objective(s):

Though each Certified Agile Leadership course varies depending on the instructor, all Certified Agile Leadership courses intend to create awareness of, and begin the journey toward, Agile Leadership.

Graduates will receive the Certified Agile Leadership (CAL 1) designation.

See Scrum Alliance Website for further details.

Agenda: Agenda (Training Details)

We create a highly interactive dynamic training environment. Each of you are unique – and so is each training. Although the essentials will be covered in every class, you will be involved in shaping the depth and focus of our time together. Each learning module is treated as a User Story (see photo) and we will co-create a unique learning journey that supports everyone’s needs.

The training will draw from the learning areas identified in the overview diagram.

Organizational Culture

“If you do not manage culture, it manages you, and you may not even be aware of the extent to which this is happening.” – Edgar Schein

  • Why Culture? Clarify why culture is critical for Organizational Success.
  • Laloux Culture Model: Discuss the Laloux culture model that will help us clarify current state and how to understand other organizations/models.
  • Agile Culture: Explore how Agile can be seen as a Culture System.
  • Agile Adoption & Transformation: Highlight differences between Agile Adoption and Transformation.
  • Dimensions of Culture: Look at key aspects of culture from “Reinventing Organizations”. Where are we and where might we go?
  • Culture Case Studies: Organizational Design: Explore how leading companies use innovative options to drive cultural operating systems.
Leadership & Organizational Models
  • Theory X – Theory Y: Models of human behaviour that are implicit in various types of management systems.
  • Management Paradigms: Contrast of Traditional “Modern” Management practices with Knowledge worker paradigm.
  • The Virtuous Cycle: Key drivers of success emergent across different high-performance organizational systems.
  • Engagement (Gallup): Gallup has 12 proven questions linked to employee engagement. How can we move the needle?
  • Advice Process: More effective decision-making using Advice Process. Build leaders. Practice with advice cards.
  • Teal Organizations: Explore what Teal Organizations are like.
Leadership Development
  • Leading Through Culture: How to lead through culture so that innovation and engagement can emerge.
  • VAST – Showing up as Leaders: VAST (Vulnerability, Authentic connection, Safety, & Trust) guides us in showing up as more effective leaders.
  • Temenos Trust Workshop: Build trust and charter your learning journey. Intro version of 2 day retreat.
  • Compassion Workshop: How to Use Compassion to Transform your Effectiveness.
  • Transformational Leadership: See how we may “be the change we want to see” in our organizations.
  • Leading Through Context: How to lead through context so that innovation and engagement can emerge.
  • Leadership in Hierarchy: Hierarchy impedes innovation. Listening and language tips to improve your leadership.
Organizational Growth
  • Working With Culture: Given a Culture Gap. What moves can we make? Work with Culture or Transformation.
  • Complex Systems Thinking: Effective change is possible when we use a Complex Systems model. Cynefin. Attractors. Emergent Change.
  • Healthy “Agile” Initiatives: How to get to a healthy initiative. How to focus on the real goals of Agile and clarify WHY.
  • People-Centric Change: The methods we use to change must be aligned with the culture we hope to foster. How we may change in a way that values people.
  • Transformation Case Study: Walkthrough of how a transformation unfolded with a 100 person internal IT group.
Audience: There are two main audiences that are addressed by this training: organizational leaders and organizational coaches. The principles and practices of organizational culture and leadership are the same regardless of your role. Organizational leaders include executives, vice presidents, directors, managers and program leads. Organizational coaches include Agile coaches, HR professionals, management consultants and internal change leaders. “The only thing of real substance that leaders do is to create and manage culture.” – Edgar Schein Facilitator(s): Michael_Sahota Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Announcement: New Leadership Training – First in Canada! appeared first on Agile Advice.

Categories: Blogs

scikit-learn: Clustering and the curse of dimensionality

Mark Needham - Sat, 08/27/2016 - 22:32

In my last post I attempted to cluster Game of Thrones episodes based on character appearances without much success. After I wrote that post I was flicking through the scikit-learn clustering documentation and noticed the following section which describes some of the weaknesses of the K-means clustering algorithm:

Inertia is not a normalized metric: we just know that lower values are better and zero is optimal.

But in very high-dimensional spaces, Euclidean distances tend to become inflated (this is an instance of the so-called “curse of dimensionality”).

Running a dimensionality reduction algorithm such as PCA prior to k-means clustering can alleviate this problem and speed up the computations.

Each episode has 638 dimensions so this is probably the problem we’re seeing. I actually thought the ‘curse of dimensionality’ referred to the greater than linear increase in computation time; I hadn’t realised it could also impact the clustering itself.

As the documentation notes, the K-Means algorithm calculates euclidean distances to work out which cluster episodes should go in. Episodes in the same cluster should have a small euclidean distance and items in different clusters should have larger ones.

I created a little script to help me understand the curse of dimensionality. I’ve got 4 pairs of vectors, of size 4, 6, 100, and 600. Half of the items in the vector match and the other half differ. I calculate the cosine similarity and euclidean distance for each pair of vectors:

from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
def distances(a, b):
    return np.linalg.norm(a-b), cosine_similarity([a, b])[0][1]
def mixed(n_zeros, n_ones):
    return np.concatenate((np.repeat([1], n_ones), np.repeat([0], n_zeros)), axis=0)
def ones(n_ones):
    return np.repeat([1], n_ones)
print distances(mixed(2, 2), ones(4))
print distances(mixed(3, 3), ones(6))
print distances(mixed(50, 50), ones(100))
print distances(mixed(300, 300), ones(600))
(1.4142135623730951, 0.70710678118654746)
(1.7320508075688772, 0.70710678118654768)
(7.0710678118654755, 0.70710678118654757)
(17.320508075688775, 0.70710678118654746)

The euclidean distance for the 600 item vector is 17x larger than for the one containing 4 items despite having the same similarity score.

Having convinced myself that reducing the dimensionality of the vectors could make a difference I reduced the size of the episodes vectors using the the Truncated SVD algorithm before trying K-means clustering again.

First we reduce the dimensionality of the episodes vectors:

from sklearn.decomposition import TruncatedSVD
n_components = 2
reducer = TruncatedSVD(n_components=n_components)
new_all = reducer.transform(all)
print("%d: Percentage explained: %s\n" % (n_components, reducer.explained_variance_ratio_.sum()))
2: Percentage explained: 0.124579183633

I’m not sure how much I should be reducing the number of dimensions so I thought 2 would an interesting place to start. I’m not sure exactly what the output of the reducer.explained_variance_ratio_ function means so I need to do some more reading to figure out whether it makes sense to carry on with a dimension of 2.

For now though let’s try out the clustering algorithm again and see how it gets on:

from sklearn.cluster import KMeans
for n_clusters in range(2, 10):
    km = KMeans(n_clusters=n_clusters, init='k-means++', max_iter=100, n_init=1)
    cluster_labels = km.fit_predict(new_all)
    silhouette_avg = metrics.silhouette_score(new_all, cluster_labels, sample_size=1000)
    print n_clusters, silhouette_avg
2 0.559681096025
3 0.498456585461
4 0.524704352941
5 0.441580592398
6 0.44703058946
7 0.447895331824
8 0.433698007009
9 0.459874485986

This time out silhouette scores are much better. I came across a tutorial from the Guide to Advanced Data Analysis which includes a table explaining how to interpret this score:

2016 08 27 21 18 14

We have a couple of cluster sizes which fit in the ‘reasonable structure’ and a few just on the edge of fitting in that category.

I tried varying the number of dimensions and found that 3 worked reasonably well, but after that the silhouette score dropped rapidly. Once we reach 30 dimensions the silhouette score is almost the same as if we hadn’t reduced dimensionality at all.

I haven’t figured out a good way of visualising the results of my experiments where I vary the dimensions and number of clusters so that’s something to work on next. I find it quite difficult to see what’s going on by just staring at the raw numbers.

I also need to read up on the SVD algorithm to understand when it is/isn’t acceptable to reduce dimensions and how much I should be reducing them by.

Any questions/thoughts/advice do let me know in the comments.

Categories: Blogs

Quotable Quotes: Leadership is the key to driving change

Learn more about transforming people, process and culture with the Real Agility Program

Jerry Doucett 201601 white background - head - 275 square“Leadership is the key to driving change and progress.  Executives and managers of Scrum teams need to nurture the environment, let go of the “how”, allow the team to learn from mistakes, and encourage and coach the growth of the collective team knowledge and overall experience.

Understanding the dramatic impact leadership has on a transitioning team is also very critical, as a single word or direction from the executive level can single-handedly affect (either positively or negatively) the team’s future behaviours and resulting successes or failures.  And without a true environment of trust built by the leadership, team members will often shy away from taking a risk to try something new or unknown.” (By Senior Agile Coach Jerry Doucett)

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Quotable Quotes: Leadership is the key to driving change appeared first on Agile Advice.

Categories: Blogs

Article Review: Obstacles to Enterprise Agility

Learn more about transforming people, process and culture with the Real Agility Program

Michael James, Certified Scrum Trainer, shares an article listing his understanding of the key obstacles to Enterprise Agility which can be found in the link on this site. He lists seven obstacles and the most meaningful seems to be number seven, staying committed to the transformation. Each individual on the team must be committed to the transformation, to be willing to endure the “storming period” which a team goes through when they are learning to work together in an agile way.

When they stay committed, as Michael James describes, then they are well on their way to adapting the agile methodologies which will allow for high-performing teams.

Experts in the field will be well aware of this concept by now, but for beginners it is worth breaking down into bits. Every conversation about agility in an organization ultimately involves a whole team changing – and not just one or two members by the way — so that an entirely new and more productive environment can allow for more efficient delivery of product.

Have you seen an agile team go through storming? What was it like? Did you see positives come out of it? Please describe your experiences here.


Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Article Review: Obstacles to Enterprise Agility appeared first on Agile Advice.

Categories: Blogs

Links for 2016-08-26 []

Zachariah Young - Sat, 08/27/2016 - 09:00
Categories: Blogs

Quotable Quote: Scrum Master has authority over the process but not over the team

Learn more about transforming people, process and culture with the Real Agility Program

Jerry Doucett 201601 white background - head - 275 squareHaving a good process is only part of the equation.  A good Scrum Master will champion and enforce that process, protect the team, encourage collaboration, highlight (escalate when necessary) and encourage the removal of obstacles, facilitate discussions, provide fair and constructive feedback, cultivate a culture of continuous improvement and learning, and work to help the team live the Agile values.

Remember that the Scrum Master has authority over the process but not over the team.  As the process champion the Scrum Master may sometimes even find themselves in a conflict between maintaining the Scrum rules and guiding the team as they discover the need to adapt practices to better align with their own needs and ways of working.  In that regard a Scrum Master should understand and embrace the servant leader role.  In the end, a Scrum Master needs to be the person that helps the team make decisions, but not the person that makes decisions for them. (By Senior Agile Coach Jerry Doucett)

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Quotable Quote: Scrum Master has authority over the process but not over the team appeared first on Agile Advice.

Categories: Blogs

Link: Software Team Development Tips

Learn more about transforming people, process and culture with the Real Agility Program

Ben Yehoolda, author of two excellent articles on LinkedIn’s Pulse, has recently connected with BERTEIG and I am happy to share these insightful pieces which offer steps to success for software development teams.

The first one, Software Development Team Composition, offers a cost-effective strategy for having a high-performing team without high costs. He writes, “Rather than building a costly team made up of only the best, the leading parameter which should dictate team composition is the complexity of the work. For the average software project the bulk of the work could be handled quite well by a B (or intermediate) level developer. The more complex work (design patterns, architecture changes, frameworks research) constitute a smaller portion of time but would definitely require an A (or Senior) level developer. To make the best use of your development budget, you should keep in mind that every development project has some very simple yet time consuming work which could be done be a C (or Junior), such as; code clean up, commenting, adding disclaimers, building unit test and small GUI alignments.” In the second article,  Steps to Building a Software Team, he writes about three steps to keep a software team engaged. This statement caught my attention. He writes, “The challenging wizardry-like act of leading a development team requires knowing more about the tools and craft of software development than the other team members. At the same time, every good leader should have the drive and charisma of a top-tier sales person to motivate the team. These two sets of skills can be hard to find in the same person. If you’re struggling to find an external candidate with these qualifications, consider looking within the team to promote or expand the responsibilities of an existing employee. Keep in mind, however, leadership training to a senior developer might work better than technology training to junior managers.”  These are great considerations for software development teams. Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post Link: Software Team Development Tips appeared first on Agile Advice.

Categories: Blogs

JIRA and Kanban vs. LeanKit

JIRA, Kanban Boards, and LeanKit
Kanban is used across many types of organizations to visualize work, limit...

The post JIRA and Kanban vs. LeanKit appeared first on Blog | LeanKit.

Categories: Companies

The Small Diet Coke: Coupling and Requirements

NetObjectives - Fri, 08/26/2016 - 14:29
When considering the quality of a design one of the things we focus on is the coupling between entities.  Coupling is both necessary (you create a system’s behavior by coupling entities to each other for collaboration) and often problematic (excessive or illogical coupling produces unpredictable side effects when changes are made). So, clearly, some coupling is good and some coupling is bad. ...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]
Categories: Companies

Conference Thoughts

Growing Agile - Fri, 08/26/2016 - 14:13
This week I attended an Agile Conference. I’m lucky enough to have attended many of these over the last 6 years. Some were amazing and some were lacking. As someone who has organised a conference before I know that this is no easy task. And everybody has their own ideas about who, how and what. […]
Categories: Companies

scikit-learn: Trying to find clusters of Game of Thrones episodes

Mark Needham - Fri, 08/26/2016 - 00:07

In my last post I showed how to find similar Game of Thrones episodes based on the characters that appear in different episodes. This allowed us to find similar episodes on an episode by episode basis, but I was curious whether there were groups of similar episodes that we could identify.

scikit-learn provides several clustering algorithms that can run over our episode vectors and hopefully find clusters of similar episodes. A clustering algorithm groups similar documents together, where similarity is based on calculating a ‘distance’ between documents. Documents separated by a small distance would be in the same cluster, whereas if there’s a large distance between episodes then they’d probably be in different clusters.

The simplest variant is K-means clustering:

The KMeans algorithm clusters data by trying to separate samples in n groups of equal variance, minimizing a criterion known as the inertia or within-cluster sum-of-squares. This algorithm requires the number of clusters to be specified.

The output from the algorithm is a list of labels which correspond to the cluster assigned to each episode.

Let’s give it a try on the Game of Thrones episodes. We’ll start from the 2 dimensional array of episodes/character appearances that we created in the previous post.

>>> all.shape
(60, 638)
>>> all
array([[0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0]])

We have a 60 (episodes) x 638 (characters) array which we can now plug into the K-means clustering algorithm:

>>> from sklearn.cluster import KMeans
>>> n_clusters = 3
>>> km = KMeans(n_clusters=n_clusters, init='k-means++', max_iter=100, n_init=1)
>>> cluster_labels = km.fit_predict(all)
>>> cluster_labels
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 2, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 0, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], dtype=int32)

cluster_labels is an array containing a label for each episode in the all array. The spread of these labels is as follows:

>>> import numpy as np
>>> np.bincount(cluster_labels)
array([19, 12, 29])

i.e. 19 episodes in cluster 0, 12 in cluster 1, and 29 in cluster 2.

How do we know if the clustering is any good?

Ideally we’d have some labelled training data which we could compare our labels against, but since we don’t we can measure the effectiveness of our clustering by calculating inter-centroidal separation and intra-cluster variance.

i.e. how close are the episodes to other episodes in the same cluster vs how close are they to episodes in the closest different cluster.

scikit-learn gives us a function that we can use to calculate this score – the silhouette coefficient.

The output of this function is a score between -1 and 1.

  • A score of 1 means that our clustering has worked well and a document is far away from the boundary of another cluster.
  • A score of -1 means that our document should have been placed in another cluster.
  • A score of 0 means that the document is very close to the decision boundary between two clusters.

I tried calculating this coefficient for some different values of K. This is what I found:

from sklearn import metrics
for n_clusters in range(2, 10):
    km = KMeans(n_clusters=n_clusters, init='k-means++', max_iter=100, n_init=1)
    cluster_labels = km.fit_predict(all)
    silhouette_avg = metrics.silhouette_score(all, cluster_labels, sample_size=1000)
    sample_silhouette_values = metrics.silhouette_samples(all, cluster_labels)
    print n_clusters, silhouette_avg
2 0.0798610142955
3 0.0648416081725
4 0.0390877994786
5 0.020165277756
6 0.030557856406
7 0.0389677156458
8 0.0590721834989
9 0.0466170527996

The best score we manage here is 0.07 when we set the number of clusters to 2. Even our highest score is much lower than the lowest score on the documentation page!

I tried it out with some higher values of K but only saw a score over 0.5 once I put the number of clusters to 40 which would mean 1 or 2 episodes per cluster at most.

At the moment our episode arrays contain 638 elements so they’re too long to visualise on a 2D silhouette plot. We’d need to apply a dimensionality reduction algorithm before doing that.

In summary it looks like character co-occurrence isn’t a good way to cluster episodes. I’m curious what would happen if we flip the array on its head and try and cluster the characters instead, but that’s for another day.

If anyone spots anything that I’ve missed when reading the output of the algorithm let me know in the comments. I’m just learning by experimentation at the moment.

Categories: Blogs

A View From the Stands: Open Agile Conference, Toronto 2015

Learn more about transforming people, process and culture with the Real Agility Program


Valerie Senyk 20150505 white background - head - 275 squareI was (wo)manning the Berteig booth for most of this day-long event with my colleague Nima Honarmandan, since we were one of the Open Agile Conference sponsors. Still, I was able to nip away and take in a seminar called “Value: From Meh to Wow” delivered by Mike Edwards, author of

After a personal introductory story about his dog dying while he was away from home, and WestJet Corporation’s remarkable assistance to him to get home as soon as possible, he listed three kinds of value: that which is monetized, that which is frugal and that which has a wow factor.

He believes WestJet has the wow factor because people are not just numbers or resources to them – people are truly people. He said that the employees of WestJet are empowered to act as if they’re owners, and so can make important (and compassionate) decisions for people on a case by case basis.

Edwards feels companies need to know what their customers’ values are, and allow themselves to align with them. Companies cannot hope to “wow” people with freebies. His point was that to create a wow factor in one’s business one needs to focus on relationships.

In an exercise, he had attendees make 3 columns on a sheet of paper. The first column was to list our employers’ values, the second to list our own values, and the third to list our customers’ values. I was “wowed” to see that, as regards my own employment and our customers, there was a great degree of alignment between all three groups, valuing such things as learning, honesty, encouragement, responsiveness and agility.

As for most of my day in the stands (at the Berteig booth), I observed that agilists (practitioners of Agile) are, by and large, very caring and user-friendly people. Between seminar sessions, hundred of them flowed through the hallways. Many of them greeted each other like long-lost buddies with big hugs, many engaged in in-depth conversations, and most were joyful and energetic.

As my colleague Nima and I met people at our booth, responded to questions, and handed out packs of Estimation Cards (freebies are fun at an event like this), I mused on the blessing of human contact. As wisdom would have it, there is a time for all aspects of life: to work, to learn, to rest, and a time to enjoy the diversity of our human family.

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!

The post A View From the Stands: Open Agile Conference, Toronto 2015 appeared first on Agile Advice.

Categories: Blogs

SAFe Lean-Agile Principles #2 and #9 updated

Agile Product Owner - Thu, 08/25/2016 - 20:25

Hi Folks,

As you can probably imagine, we are quite relieved to have finally published the SAFe 4.0 Reference Guide. But after our five-minute celebration we went right back to work on some other projects, one of which is another SAFe book that Richard and I are collaborating on. The work has revealed some room for improvement on the SAFe Lean-Agile Principles articles, so I’m now in the process of getting those updated. They aren’t long, but distilling the nine most valuable principles into these short articles isn’t a trivial effort either.

I’ve largely completed that work, and in the interest of accelerating value delivery, I’ve already pushed two of them to the site. Specifically, Principle 2: Apply Systems Thinking, and Principle 9: Decentralize Decision-making. These aren’t dramatically different conceptually, but they are a little deeper, a little better written, and hopefully, a little more valuable.

Stay SAFe!

—Dean and the Framework team

Categories: Blogs

Knowledge Sharing

SpiraTeam is a agile application lifecycle management (ALM) system designed specifically for methodologies such as scrum, XP and Kanban.