Skip to content

Blogs

If Showcasing Technical Work, Then Remember to Keep Stakeholders Engaged

Learn more about our Scrum and Agile training sessions on WorldMindware.com

During Sprint Reviews, I’ve observed teams showcasing the results of purely technical work. Sometimes the resolution of a defect caused by some form of technical debt (“we noticed that we were making too many service calls”); or something related to development infrastructure (“we had to switch to a new testing framework”.) In such cases, the presentation frequently jumps quickly into a description of what was done showing, for example, the before and after behaviour connected to the defect or the results produced with a testing framework.

Potential problems with this are that business stakeholders attending the meeting:

  • might be left wondering what this means to them
  • feel unable to provide any feedback
  • and in the worst case scenario, might feel they’ve wasted their time attending the Review.

A team might ask themselves, “what is the reason for us to have purely technical items in our Product Backlog?” But leaving that aside, the perils of losing stakeholder engagement during the Sprint Review could be mitigated by framing the presentation around the following questions:

  • How is the world better now that we’ve fixed this defect?
  • What’s the impact on the users and the business?
  • What specific new capability does this technical work allow the team to do now for the business they serve (that they couldn’t do before)?

Submitted by Fernando Cuenca

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to informationPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post If Showcasing Technical Work, Then Remember to Keep Stakeholders Engaged appeared first on Agile Advice.

Categories: Blogs

Reducing the Load on the Product Owner

Agile For All - Bob Hartman - Sat, 07/18/2015 - 21:57

jeff-patton-no-time-product-owner

Product Owners have it tough. They need to spend time and energy working to understand the needs of customers and users. They need to work with the development team to prioritize and decompose Product Backlog Items into small user stories with acceptance criteria. Finally, they need to work with key stakeholders within the business to get budget, garner support for their efforts, and align on how the work of their scrum team aligns with the larger vision and strategy of the organization. Not only does this create a tremendous amount of stress for the Product Owner, but it leaves no slack time. Slack is a key ingredient for creativity and growth, and without that, innovation, value delivery, and personal well being suffer.

At a recent session, I along with my Agile For All colleagues Peter Saddington and Richard Lawrence helped a group to use Lean Thinking, the Theory of Constraints, and the Product Owner Responsibility Matrix to reduce the load on their product owners. Here is our approach.

Exploit the Constraint

The Theory Of Constraints recommends the use of Five Focusing Steps to optimize any system:

five-focusing-steps

While there are some standard steps to doing number 1: Identify the Constraint, in many teams using Scrum, the Product Owner is clearly the constraint, so we will jump right to step 2: Exploit the Constraint.

While the term exploit usually has a negative connotation when referring to humans, in this case, “Exploiting the Constraint” simply means making it as effective as possible, since it is the limiting factor on the speed of the entire system. In many cases, there are ways to reduce the amount of time required by the Product Owner for certain responsibilities. Any time we can free up from the Product Owner immediately impacts their ability to work on more important things, immediately impacting the speed and effectiveness of the entire system.

Enter the Matrix

matrix-pills

No, not that Matrix.

The Product Owner Responsibility Matrix is a thinking tool for helping a Product Owner to focus on only the highest value items on their plate. We can look at every activity that is a current responsibility for a Product Owner and assign it to one of five categories:

  1. Do Less – an activity that is not adding value in the organization. If so, work on doing it less or not at all.
  2. Delegate Completely – an activity that can be safely delegated to anyone else in the organization. They are by definition not the constraint.
  3. Delegate and Mentor – an activity that can be delegated but that would still benefit from some mentoring by the Product Owner. This reduces the amount of time spent by the Product Owner as well as growing expertise in others.
  4. Collaborate – an activity that would achieve a higher quality outcome by involving the team or stakeholders. While the initial time spent in collaboration might not be the same or higher, the improved result means there will be less rework. It is likely that those collaborating will also be more enrolled in the result, reducing time spent by theProduct Owner in explaining the what and why of a decision.
  5. Own – an activity that really must be done by the Product Owner.
Using the Product Owner Responsibility Matrix

To use the matrix, we recommend a facilitated activity involving theProduct Owner as well as team members and possibly other key stakeholders in the organization.

Step One: Brainstorm Current Responsibilities

We find it is helpful to use a framework for brainstorming responsibilities. One such framework is to look at the typical three key stakeholders with whom a Product Owner interacts: Users/Customers, The Business, and The Team. Label three flip chart pages with one of the three stakeholders each, and put them on the wall in three corners of the room. Ask people to stand by the flip chart that they know the most about, and have them use sticky notes to brainstorm all of the activities that the Product Owner currently performs with that stakeholder group. Have them write one activity on each sticky note. For example, on the “Team” poster, you may see sticky notes labeled “Write User Stories”, “Define Acceptance Tests”, and “Participate in the Scrum Ceremonies”, among several others.

Once the activities are all on sticky notes, have the team group stickies on their poster that seem related, and then label each group with a category name, creating a list of categories on the left hand side of the poster. At this point, have each group share the categories and the stickies in each category and ask for feedback from the other two groups about anything they may have overlooked.

Step Two: Add the Matrix

On the right hand side of the poster, draw a grid with five columns and label them 1,2,3,4, and 5, which correspond to the matrix options (1:Do Less through 5:Own). Make the columns about the width of one sticky note. Have the groups move the sticky notes from each category over to one of the five columns. Here is a sample of what the Business poster looked like in a recent session, about half way through the activity:

po-biz-poster

Step 3: Prioritize the Stickies

At this point, you should have a good list of activities that can be offloaded from the Product Owner in some way. Create a backlog of the ones that involve a change, and prioritize them based on the Product Owner’s estimate of how much time it would save them. You may also include how difficult it would be to make the change in the prioritization discussion, as some changes may require some work to put in to place. Stickies in the “Do Less” column may especially require some discussions with others in the organization to get buy in.

Step 4: Work on your Backlog!

Now you’ve got a nice backlog of items that you can start working down to free up time for your Product Owner. Each item you can complete will help you exploit the constraint, ensuring that the Product Owner is focusing their time only on the most critical, value add work that only they can do.

Learning More

If you are interested in talking with us to learn more about this technique and other ways to help Product Owners be their most awesome selves, please contact ussign up from an upcoming Certified Scrum Product Owner course, or check out our online course 80/20 Product Ownership.

 

The post Reducing the Load on the Product Owner appeared first on Agile For All.

Categories: Blogs

Video: Frustrated? It is probably your fault

Thought Nursery - Jeffrey Fredrick - Sat, 07/18/2015 - 12:46

I had great fun with my Devopsdays Amsterdam talk and the video is now online, joining the slides that I posted previously.

The last time I spoke at a Devopsdays was London 2013 (video here). That was another fun talk, and had some overlap in content, but I did feel that I tried to put too many concepts in a single 30 minute talk. My goal this time was to be much more deliberate and leave enough time for each concept. Where I ended up is a talk in three parts. Part one is cognitive psychology, how our mind generates an illusion of certainty where we don’t deserve it. Part two is Action Science and the Mutual Learning Model as a set of behaviours that appropriate for an uncertain world. Part three is the need for practice. This section uses the piano analogy and then — my big risk! — a live demonstration. A brave member of the audience joined me on stage to try applying the concepts I’d just discussed.

As you sit and watch this video I hope it is the final of section of the talk that makes the biggest impression. All the video watching, all the reading, all the learning will mean nothing if you don’t act, if you don’t practice and find the limits of your current abilities and then learn to move beyond them. And I’ll make the first step easy: Download the slides and try the exercise in the video. What would you say to Ted? Write it out and read it aloud. How did you do? Maybe you want to try again…

(Lots more great talks online at the devopsdays vimeo account.)

Categories: Blogs

How To Get Smarter By Making Distinctions

J.D. Meier's Blog - Sat, 07/18/2015 - 09:17

"Whatever you do in life, surround yourself with smart people who'll argue with you." -- John Wooden

There’s a very simple way to get smarter.

You can get smarter by creating categories.

Not only will you get smarter, but you’ll also be more mindful, and you’ll expand your vocabulary, which will improve your ability to think more deeply about a given topic or domain.

In my post, The More Distinctions You Make, the Smarter You Get, I walk through the ins and outs of creating categories to increase your intelligence, and I use the example of “fat.”   I attempt to show how “Fat is bad” isn’t very insightful, and how by breaking “fat” down into categories, you can dive deeper and reveal new insight to drive better decisions and better outcomes.

I’m this post, I’m going to walk this through with an example, using “security” as the topic.

The first time I heard the word “security”, it didn’t mean much to me, beyond “protect.”

The next thing somebody taught me, was how I had to focus on CIA:  Confidentiality, Integrity, and Availability.

That was a simple way to break security down into meaningful parts.

And then along came Defense in Depth.   A colleague explained that Defense in Depth meant thinking about security in terms of multiple layers:  Network, Host, Application, and Data.

But then another colleague said, the real key to thinking about security and Defense in Depth, was to think about it in terms of people, process, and technology.

As much as I enjoyed these thought exercises, I didn’t find them actionable enough to actually improve software or application security.  And my job was to help Enterprise developers build better Line-Of-Business applications that were scalable and secure.

So our team went to the drawing board to map out actionable categories to take application security much deeper.

Right off the bat, just focusing on “application” security vs. “network” security or “host” security, helped us to get more specific and make security more tangible and more actionable from an Line-of-Business application perspective.

Security Categories

Here are the original security categories that we used to map out application security and make it more actionable:

  1. Input and Data Validation
  2. Authentication
  3. Authorization
  4. Configuration Management
  5. Sensitive Data
  6. Session Management
  7. Cryptography
  8. Exception Management
  9. Auditing and Logging

Each of these buckets helped us create actionable principles, patterns, and practices for improving security.

Security Categories Explained

Here is a brief description of each application security category:

Input and Data Validation
How do you know that the input your application receives is valid and safe? Input validation refers to how your application filters, scrubs, or rejects input before additional processing. Consider constraining input through entry points and encoding output through exit points. Do you trust data from sources such as databases and file shares?

Authentication
Who are you? Authentication is the process where an entity proves the identity of another entity, typically through credentials, such as a user name and password.

Authorization
What can you do? Authorization is how your application provides access controls for resources and operations.

Configuration Management
Who does your application run as? Which databases does it connect to? How is your application administered? How are these settings secured? Configuration management refers to how your application handles these operational issues.

Sensitive Data
How does your application handle sensitive data? Sensitive data refers to how your application handles any data that must be protected either in memory, over the network, or in persistent stores.

Session Management
How does your application handle and protect user sessions? A session refers to a series of related interactions between a user and your Web application.

Cryptography
How are you keeping secrets (confidentiality)? How are you tamper-proofing your data or libraries (integrity)? How are you providing seeds for random values that must be cryptographically strong? Cryptography refers to how your application enforces confidentiality and integrity.

Exception Management
When a method call in your application fails, what does your application do? How much do you reveal? Do you return friendly error information to end users? Do you pass valuable exception information back to the caller? Does your application fail gracefully?

Auditing and Logging
Who did what and when? Auditing and logging refer to how your application records security-related events.

As you can see, just by calling out these different categories, you suddenly have a way to dive much deeper and explore application security in depth.

The Power of a Security Category

Let’s use a quick example.  Let’s take Input Validation.

Input Validation is a powerful security category, given how many software security flaws and how many vulnerabilities and how many attacks all stem from a lack of input validation, including Buffer Overflows.

But here’s the interesting thing.   After quite a bit of research and testing, we found a powerful security pattern that could help more applications stand up to more security attacks.  It boiled down to the following principle:

Validate for length, range, format, and type.

That’s a pithy, but powerful piece of insight when it comes to implementing software security.

And, when you can’t validate the input, make it safe by sanitizing the output.  And along these lines, keep user input out of the control path, where possible.

All of these insights flow from just focusing on Input Validation as a security category.

Threats, Attacks, Vulnerabilities, and Countermeasures

Another distinction our team made was to think in terms of threats, attacks, vulnerabilities, and countermeasures.  We knew that threats could be intentional and malicious (as in the case of attacks), but they could also be accidental and unintended.

We wanted to identify vulnerabilities as weaknesses that could be addressed in some way.

We wanted to identify countermeasures as the actions to take to help mitigate risks, reduce the attack surface, and address vulnerabilities.

Just by chunking up the application security landscape into threats, attacks, vulnerabilities, and countermeasures, we empowered more people to think more deeply about the application security space.

Security Vulnerabilities Organized by Security Categories

Using the security categories above, we could easily focus on finding security vulnerabilities and group them by the relevant security category.

Here are some examples:

Input/Data Validation

  • Using non-validated input in the Hypertext Markup Language (HTML) output stream
  • Using non-validated input used to generate SQL queries
    Relying on client-side validation
  • Using input file names, URLs, or user names for security decisions
  • Using application-only filters for malicious input
  • Looking for known bad patterns of input
  • Trusting data read from databases, file shares, and other network resources
  • Failing to validate input from all sources including cookies, query string parameters, HTTP headers, databases, and network resources

Authentication

  • Using weak passwords
  • Storing clear text credentials in configuration files
  • Passing clear text credentials over the network
  • Permitting over-privileged accounts
  • Permitting prolonged session lifetime
  • Mixing personalization with authentication

Authorization

  • Relying on a single gatekeeper
  • Failing to lock down system resources against application identities
  • Failing to limit database access to specified stored procedures
  • Using inadequate separation of privileges

Configuration Management

  • Using insecure administration interfaces
  • Using insecure configuration stores
  • Storing clear text configuration data
  • Having too many administrators
  • Using over-privileged process accounts and service accounts

Sensitive Data

  • Storing secrets when you do not need to
  • Storing secrets in code
  • Storing secrets in clear text
  • Passing sensitive data in clear text over networks

Session Management

  • Passing session identifiers over unencrypted channels
  • Permitting prolonged session lifetime
  • Having insecure session state stores
  • Placing session identifiers in query strings

Cryptography

  • Using custom cryptography
  • Using the wrong algorithm or a key size that is too small
  • Failing to secure encryption keys
  • Using the same key for a prolonged period of time
  • Distributing keys in an insecure manner

Exception Management

  • Failing to use structured exception handling
  • Revealing too much information to the client

Auditing and Logging

  • Failing to audit failed logons
  • Failing to secure audit files
  • Failing to audit across application tiers
Threats and Attacks Organized by Security Categories

Again, using our security categories, we could then group threats and attacks by relevant security categories.

Here are some examples of security threats and attacks organized by security categories:

Input/Data Validation

  • Buffer overflows
  • Cross-site scripting
  • SQL injection
  • Canonicalization attacks
  • Query string manipulation
  • Form field manipulation
  • Cookie manipulation
  • HTTP header manipulation

Authentication

  • Network eavesdropping
  • Brute force attacks
  • Dictionary attacks
  • Cookie replay attacks
  • Credential theft

Authorization

  • Elevation of privilege
  • Disclosure of confidential data
  • Data tampering
  • Luring attacks

Configuration Management

  • Unauthorized access to administration interfaces
  • Unauthorized access to configuration stores
  • Retrieval of clear text configuration secrets
  • Lack of individual accountability

Sensitive Data

  • Accessing sensitive data in storage
  • Accessing sensitive data in memory (including process dumps)
  • Network eavesdropping
  • Information disclosure

Session Management

  • Session hijacking
  • Session replay
  • Man-in-the-middle attacks

Cryptography

  • Loss of decryption keys
  • Encryption cracking

Exception Management

  • Revealing sensitive system or application details
  • Denial of service attacks

Auditing and Logging

  • User denies performing an operation
  • Attacker exploits an application without trace
  • Attacker covers his tracks
Countermeasures Organized by Security Categories

Now here is where the rubber really meets the road.  We could group security countermeasures by security categories to make them more actionable.

Here are example security countermeasures organized by security categories:

Input/Data Validation

  • Do not trust input
  • Validate input: length, range, format, and type
  • Constrain, reject, and sanitize input
  • Encode output

Authentication

  • Use strong password policies
  • Do not store credentials
  • Use authentication mechanisms that do not require clear text credentials to be passed over the network
  • Encrypt communication channels to secure authentication tokens
  • Use HTTPS only with forms authentication cookies
  • Separate anonymous from authenticated pages

Authorization

  • Use least privilege accounts
  • Consider granularity of access
  • Enforce separation of privileges
  • Use multiple gatekeepers
  • Secure system resources against system identities

Configuration Management

  • Use least privileged service accounts
  • Do not store credentials in clear text
  • Use strong authentication and authorization on administrative interfaces
  • Do not use the Local Security Authority (LSA)
  • Avoid storing sensitive information in the Web space
  • Use only local administration

Sensitive Data

  • Do not store secrets in software
  • Encrypt sensitive data over the network
  • Secure the channel

Session Management

  • Partition site by anonymous, identified, and authenticated users
  • Reduce session timeouts
  • Avoid storing sensitive data in session stores
  • Secure the channel to the session store
  • Authenticate and authorize access to the session store

Cryptography

  • Do not develop and use proprietary algorithms (XOR is not encryption. Use platform-provided cryptography)
  • Use the RNGCryptoServiceProvider method to generate random numbers
  • Avoid key management. Use the Windows Data Protection API (DPAPI) where appropriate
  • Periodically change your keys

Exception Management

  • Use structured exception handling (by using try/catch blocks)
  • Catch and wrap exceptions only if the operation adds value/information
  • Do not reveal sensitive system or application information
  • Do not log private data such as passwords

Auditing and Logging

  • Identify malicious behavior
  • Know your baseline (know what good traffic looks like)
  • Use application instrumentation to expose behavior that can be monitored

As you can see, the security countermeasures can easily be reviewed, updated, and moved forward, because the actionable principles are well organized by the security categories.

There are many ways to use creating categories as a way to get smarter and get better results.

In the future, I’ll walk through how we created an Agile Security approach, using categories.

Meanwhile, check out my post on The More Distinctions You Make, the Smarter You Get to gain some additional insights into how to use empathy and creating categories to dive deeper, learn faster, and get smarter on any topic you want to take on.

Categories: Blogs

We Help Our Customers Transform

J.D. Meier's Blog - Sat, 07/18/2015 - 08:34

"Innovation—the heart of the knowledge economy—is fundamentally social." -- Malcolm Gladwell

I’m a big believer in having clarity around what you help your customers do.

I was listening to Satya Nadella’s keynote at the Microsoft Worldwide Partner Conference, and I like how he put it so simply, that we help our customers transform.

Here’s what Satya had to say about how we help our customers transform their business:

“These may seem like technical attributes, but they are key to how we drive business success for our customers, business transformation for our customers, because all of what we do, collectively, is centered on this core goal of ours, which is to help our customers transform.

When you think about any customer of ours, they're being transformed through the power of digital technology, and in particular software.

There isn't a company out there that isn't a software company.

And our goal is to help them differentiate using digital technology.

We want to democratize the use of digital technology to drive core differentiation.

It's no longer just about helping them operate their business.

It is about them excelling at their business using software, using digital technology.

It is about our collective ability to drive agility for our customers.

Because if there is one truth that we are all faced with, and our customers are faced with, it's that things are changing rapidly, and they need to be able to adjust to that.

And so everything we do has to support that goal.

How do they move faster, how do they interpret data quicker, how are they taking advantage of that to take intelligent action.

And of course, cost.

But we'll keep coming back to this theme of business transformation throughout this keynote and throughout WPC, because that's where I want us to center in on.

What's the value we are adding to the core of our customer and their ability to compete, their ability to create innovation.

And anchored on that goal is our technical ambition, is our product ambition.”

Transformation is the name of the game.

You Might Also Like

Satya Nadella is All About Customer Focus

SatyaSatya Nadella on a Mobile-First, Cloud-First World

Satya Nadella on Empower Every Person on the Planet

Satya Nadella on Everyone Has To Be a Leader

Satya Nadella on How the Key To Longevity is To Be a Learning Organization

Satya Nadella on Live and Work a Meaningful Life

Sayta Nadelle on The Future of Software

Categories: Blogs

Satya Nadella on a Mobile-First, Cloud-First World

J.D. Meier's Blog - Sat, 07/18/2015 - 08:11

You hear Mobile-First, Cloud-First all the time.

But do you ever hear it really explained?

I was listening to Satya Nadella’s keynote at the Microsoft Worldwide Partner Conference, and I like how he walked through how he thinks about a Mobile-First, Cloud-First world.

Here’s what Satya had to say:

“There are a couple of attributes.

When we talk about Mobile-First, we are talking about the mobility of the experience.

What do we mean by that?

As we look out, the computing that we are going to interface with, in our lives, at home and at work, is going to be ubiquitous.

We are going to have sensors that recognize us.

We are going to have computers that we are going to wear on us.

We are going to have computers that we touch, computers that we talk to, the computers that we interact with as holograms.

There is going to be computing everywhere.

But what we need across all of this computing, is our experiences, our applications, our data.

And what enables that is in fact the cloud acting as a control plane that allows us to have that capability to move from device to device, on any given day, at any given meeting.

So that core attribute of thinking of mobility, not by being bound to a particular device, but it's about human mobility, is very core to our vision.

Second, when we think about our cloud, we think distributed computing will remain distributed.

In fact, we think of our servers as the edge of our cloud.

And this is important, because there are going to be many legitimate reasons where people will want digital sovereignty, people will want data residency, there is going to be regulation that we can't anticipate today.

And so we have to think about a distributed cloud infrastructure.

We are definitely going to be one of the key hyper-scale providers.

But we are also going to think about how do we get computing infrastructure, the core compute, storage, network, to be distributed throughout the world.

These may seem like technical attributes, but they are key to how we drive business success for our customers, business transformation for our customers, because all of what we do, collectively, is centered on this core goal of ours, which is to help our customers transform.”

That’s a lot of insight, and very well framed for creating our future and empowering the world.

You Might Also Like

Microsoft Explained: Making Sense of the Microsoft Platform Story

Satya Nadella is All About Customer Focus

Satya Nadella on Empower Every Person on the Planet

Satya Nadella on Everyone Has To Be a Leader

Satya Nadella on How the Key To Longevity is To Be a Learning Organization

Satya Nadella on Live and Work a Meaningful Life

Sayta Nadelle on The Future of Software

Categories: Blogs

Empower Every Person on the Planet to Achieve More

J.D. Meier's Blog - Sat, 07/18/2015 - 06:32

It’s great to get back to the basics, and purpose is always a powerful starting point.

I was listening to Satya Nadella’s keynote at the Microsoft Worldwide Partner Conference, and I like how he walked through the Microsoft mission in a mobile-first, cloud-first world.

Here’s what Satya had to say:

“Our mission:  Empowering every person and every business on the planet to achieve more.

(We find that by going back into our history and re-discovering that core sense of purpose, that soul ... a PC in every home, democratizing client/server computing.)

We move forward to a Mobile-First, Cloud-First world.

We care about empowerment.

There is no other ecosystem that is primarily, and solely, built to help customers achieve greatness.

We are focused on helping our customers achieve greatness through digital technology.

We care about both individuals and organizations.  That intersection of people and organizations is the cornerstone of what we represent as excellence.

We are a global company.  We want to make sure that the power of technology reaches every country, every vertical, every organization, irrespective of size.

There will be many goals.

What remains constant is this sense of purpose, the reason why this ecosystem exists.

This is a mission that we go and exercise in a Mobile-First, Cloud-First world.”

If I think back to why I originally joined Microsoft, it was to empower every person on the planet to achieve more.

And the cloud is one powerful enabler.

You Might Also Like

Satya Nadella is All About Customer Focus

Satya Nadella on Everyone Has To Be a Leader

Satya Nadella on How the Key To Longevity is To Be a Learning Organization

Satya Nadella on Live and Work a Meaningful Life

Sayta Nadelle on The Future of Software

Categories: Blogs

R: Blog post frequency anomaly detection

Mark Needham - Sat, 07/18/2015 - 01:34

I came across Twitter’s anomaly detection library last year but haven’t yet had a reason to take it for a test run so having got my blog post frequency data into shape I thought it’d be fun to run it through the algorithm.

I wanted to see if it would detect any periods of time when the number of posts differed significantly – I don’t really have an action I’m going to take based on the results, it’s curiosity more than anything else!

First we need to get the library installed. It’s not on CRAN so we need to use devtools to install it from the github repository:

install.packages("devtools")
devtools::install_github("twitter/AnomalyDetection")
library(AnomalyDetection)

The expected data format is two columns – one containing a time stamp and the other a count. e.g. using the ‘raw_data’ data frame that is in scope when you add the library:

> library(dplyr)
> raw_data %>% head()
            timestamp   count
1 1980-09-25 14:01:00 182.478
2 1980-09-25 14:02:00 176.231
3 1980-09-25 14:03:00 183.917
4 1980-09-25 14:04:00 177.798
5 1980-09-25 14:05:00 165.469
6 1980-09-25 14:06:00 181.878

In our case the timestamps will be the start date of a week and the count the number of posts in that week. But first let’s get some practice calling the anomaly function using the canned data:

res = AnomalyDetectionTs(raw_data, max_anoms=0.02, direction='both', plot=TRUE)
res$plot

2015 07 18 00 09 22

From this visualisation we learn that we should expect both high and low outliers to be identified. Let’s give it a try with the blog post publication data.

We need to get the data into shape so we’ll start by getting a count of the number of blog posts by (week, year) pair:

> df %>% sample_n(5)
                                                           title                date
1425                            Coding: Copy/Paste then refactor 2009-10-31 07:54:31
783  Neo4j 2.0.0-M06 -> 2.0.0-RC1: Working with path expressions 2013-11-23 10:30:41
960                                        R: Removing for loops 2015-04-18 23:53:20
966   R: dplyr - Error in (list: invalid subscript type 'double' 2015-04-27 22:34:43
343                     Parsing XML from the unix terminal/shell 2011-09-03 23:42:11
 
> byWeek = df %>% 
    mutate(year = year(date), week = week(date)) %>% 
    group_by(week, year) %>% summarise(n = n()) %>% 
    ungroup() %>% arrange(desc(n))
 
> byWeek %>% sample_n(5)
Source: local data frame [5 x 3]
 
  week year n
1   44 2009 6
2   37 2011 4
3   39 2012 3
4    7 2013 4
5    6 2010 6

Great. The next step is to translate this data frame into one containing a date representing the start of that week and the number of posts:

> data = byWeek %>% 
    mutate(start_of_week = calculate_start_of_week(week, year)) %>%
    filter(start_of_week > ymd("2008-07-01")) %>%
    select(start_of_week, n)
 
> data %>% sample_n(5)
Source: local data frame [5 x 2]
 
  start_of_week n
1    2010-09-10 4
2    2013-04-09 4
3    2010-04-30 6
4    2012-03-11 3
5    2014-12-03 3

We’re now ready to plug it into the anomaly detection function:

res = AnomalyDetectionTs(data, 
                         max_anoms=0.02, 
                         direction='both', 
                         plot=TRUE)
res$plot

2015 07 18 00 24 20

Interestingly I don’t seem to have any low end anomalies – there were a couple of really high frequency weeks when I first started writing and I think one of the other weeks contains a New Year’s Eve when I was particularly bored!

If we group by month instead only the very first month stands out as an outlier:

data = byMonth %>% 
  mutate(start_of_month = ymd(paste(year, month, 1, sep="-"))) %>%
  filter(start_of_month > ymd("2008-07-01")) %>%
  select(start_of_month, n)
res = AnomalyDetectionTs(data, 
                         max_anoms=0.02, 
                         direction='both',       
                         #longterm = TRUE,
                         plot=TRUE)
res$plot

2015 07 18 00 34 02

I’m not sure what else to do as far as anomaly detection goes but if you have any ideas please let me know!

Categories: Blogs

C# 6 Exception Filters Will Improve Your Home Life

Jimmy Bogard - Fri, 07/17/2015 - 23:00

If you’re like me and you enjoy nice, peaceful weekends, but keep getting hassled when something bad happens in production to shatter the blissful silence you’re enjoying, a great new feature of C# 6 will help you out:

try {
  // Do stuff
} catch (Exception e) when (
  (DateTime.Now.DayOfWeek == DayOfWeek.Saturday)
  || (DateTime.Now.DayOfWeek == DayOfWeek.Sunday)) {
  // Swallow
}

Swallow exceptions – but only on weekends! Now you won’t get bothered any more, the app will always be up and running! The error logs will be nice and clean as if nothing has gone wrong at all.

This can be extended if it’s just one person hassling you a lot:

try {
  // Do stuff
} catch (Exception e) when (Thread.CurrentPrincipal.Identity.Name == "JBOGARD") {
  // Swallow
}

The possibilities are endless.

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

Categories: Blogs

Product Manager, Product Owner, or Business Analyst?

Johanna Rothman - Thu, 07/16/2015 - 19:44

Do you have a title such as product manager, product owner, or business analyst?

We hear  these titles all the time. What does each do?

Here is how I have seen successful agile projects and programs use people in these positions. Note that I am discussing agile projects and programs.

The product manager creates the roadmap. She has the product vision over the entire life of the product. Typically, what’s In the roadmap are larger than epics—they are themes or feature sets. 

Product management means thinking strategically about the product. You might require several projects or programs to achieve what the product manager wants as a strategic vision.

Product owners (PO) work with agile teams to translate the strategic vision into Minimum Viable Products. The PO decides when the team(s) have done enough to release. See the release frequency image to understand the cost and value of releasing.

The business analyst may do any of these things. In my experience, I have seen business analysts focus on “what does this requirement/feature/story really mean to the team and/or the product?” I have seen fewer BAs do the strategic visioning of the product over its lifetime. I have seen BAs work with POs when the PO was not available enough for the team. I have seen BAs do great work breaking stories into smaller components of value, not architectural components.

Your team might have different names for these positions. Each team needs the strategic lifetime-of-the-product view; the tactical view for the next iteration or so and the knowledge of how to re-rank the backlog; and the ability to translate stories into small valuable chunks. 

Can one person do each of these things? It depends on the person. I have found it difficult to move quickly from the tactical to strategic and back again (or vice versa). Maybe you know how. For me, that is a form of multitasking. 

The more important questions are: do you have the roles you need, at the time you need them on your team? If you are one of these people, do you know how to perform these roles? If you are outside the organization in some way, do you know what you need to do, to perform these roles?

If you don’t know what to do to help your team, consider participating in Product Owner for Agencies training. Marcus Blankenship and I will help you learn what to do, and coach you in real time as to how to do it best for your team. I hope to see you there.

Categories: Blogs

New Video: Myths of Scrum – A Public Retrospective

Learn more about our Scrum and Agile training sessions on WorldMindware.com

Although subtle, having a public retrospective can do terrible damage to a Scrum team.  In this video I explain what I mean by “public”, why it is so bad, and what you should do instead.  This is part of a video series on the Myths of Scrum that is meant to respond to some of the most common mis-information (myths) about Scrum and Scrum practices.  I will follow-up this video in several weeks with a written article complimentary to this video.  Feel free to let me know in the comments if you have any topics that you would like me to cover in my video series!

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to informationPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post New Video: Myths of Scrum – A Public Retrospective appeared first on Agile Advice.

Categories: Blogs

5 Tips for Accelerated Learning

thekua.com@work - Thu, 07/16/2015 - 10:19

I spent all of last year in Berlin, where I focused on learning the German language. After years learning different programming languages like Java, C#, Javascript and Ruby (not to mention all the tools, frameworks and libraries) and several years training and teaching I figured I could “eat my own dog food” and apply what I learnt about learning to a real language.

Enter to Learn stone signStone sign: Enter to Learn. Image take from flikr under the Creative Commons licence.

Since that year, I talked to numerous people impressed that I can speak fluent German after living in Berlin. They often ask me, how did I found my experience. This post is a good summary of what I found myself repeating. Firstly, if you really, really want to learn German, Berlin is probably not the best place since there are so many foreigners living there and the level of spoken English is ridiculously high. You can easily fall into hanging out with the English-speaking crowd, enjoy the city, and fail to pick up more than rudimentary German.

Although I have very specific tips for learning the German language, this is more focused on what helped me learn the most that I think applies to any subject.

1. Find different teachers

At the start of the year, I spent the first three months doing an immersive language course at the Goethe Institute. Over the two classes, I experienced four different teachers, each with their own style and emphasis on what is important to learn. Although I wished that I had spent less time with one of the teachers, I found their point of view was sometimes useful. Each teacher focused on different aspects and it made my learning all that richer.

Over the rest of the year, I found other avenues to teach me, guide me and give the feedback I needed to grow. If I had stuck with a single teacher, I would have missed out on a number of other valuable perspectives.

2. Have a goal, and keep focused on it

Unlike many European people, I was never bilingual as a child. Moving to England from Australia, I came to appreciate how useful a second language was by meeting many other bilingual people. Although I learned Japanese in school, I believe it is more important to have the ability to use it.As they say:

Use it, or lose it.

A whole year to learn a language was a once in a lifetime opportunity and my goal was to be fluent by the end of the year. I had a whole bunch of other personal reasons to learn German, but figured many things came into alignment and I would make the most of this opportunity.

I reminded myself throughout the year about my goal, and why it was important to me, and it helped me keep focus on the learning. It helped me in particular when the going got tough, and it will get tough!

3. Celebrate progress

After a year passed by, I met with some of my colleagues again who were impressed that I could now speak with them fluently in their native tongue. Although it’s easy to celebrate that final result, there were many times along the way that I wanted to give up because it was hard, and I felt frustrated that I felt like I wasn’t learning as fast I wanted to.

I was sharing a flat with a German, who knew that I was wanting to speak fluent German. Although he could speak English very well, he spoke only to me in German even if it would have been easier for him to communicate in English. I remember coming home one day, exasperatedly asking, “Can we just speak in English?!” His answer, “Nein!” (No, in German).

Although there were bad days, there were also good days and I made sure to sit back and acknowledge these small milestones, such as watching my first film in German with subtitles (and understanding most of it), completing my first German novel, and going to the Town Hall to register myself without speaking a word of English!

Each small step moves you towards your final goal, and celebrating progress will help you overcome the learning hurdles you experience along the way.

4. Vary your learning activities

One problem with a long-term learning goal, is that you will get bored and distracted. At the start of my year, I was naturally doing a lot of grammatical textbook exercises which was useful for the classroom. However I couldn’t see myself continuing to do just these exercises for the rest of the year. I wanted other ways to learn but would help me keep engaged. Therefore I combined learning German with other activities that I enjoyed, such as meeting with people (the German Stammtisch the language school organised was a good one), watching movies and reading books I liked in German (thanks library!), meeting with a tandem partner, and listening to the radio.

A friend gave me the book, “111 Orte in Berlin, die man gesehen haben muss” which roughly translates to, “111 must see places in Berlin.” The book has two pages for each place, one with a short description and the other with a picture and was perfect for dipping in and out. I could read about a place I wanted to visit, and because I lived in Berlin, could go and see what it was talking about. At the same time, it helped me deepen my vocabulary and help me experience the unique experiences Berlin had of offer.

Later in the year, I spent some time traveling around in Germany, where I did a number of tours in German. Each of these experiences kept the learning alive for me and I never grew bored about “learning German” because I was doing it at the same time as I was doing other activities I was enjoying.

5. Accept you’ll never be perfect

Early on in my career, I discovered the idea of the Beginner’s Mind (Shoshin). For me, part of this accepts that I do not, and cannot know everything and that means there are new things to learn. In learning German, I found that my vocabulary will never be as rich as it is in English because there are situations I have never found myself in.

A good example of this was talking to a friend of mine when we first worked in a German-speaking office. She told me this story:

Although I studied German at school and was very fluent, I was shocked the first time that I was running a project kickoff for a German-speaking client. Not only was I learning new German words for the domain (transportation) but I was also learning new German words for technology terms I had taken for granted.

Our experiences shape who we are, and we cannot possibly be experts at everything, and that’s perfectly fine. I found it was more productive to focus on getting better than worrying about how close to mastery I was.

Conclusion

After years of coaching, teaching and training technology, I have many techniques that I consider useful as learning strategies. I found last year was a great opportunity to try applying some of them to a completely different skill and see how far I could go. Happily, it seemed to have worked.

I hope you found this article useful and that you will find these tips above useful. In summary, try to:

  1. Find different teachers
  2. Have a goal, and keep focused on it
  3. Celebrate progress
  4. Vary your learning activities
  5. Accept you’ll never be perfect

What other learning strategies do you find work for you?

Categories: Blogs

Neo4j: The football transfers graph

Mark Needham - Thu, 07/16/2015 - 08:40

Given we’re still in pre season transfer madness as far as European football is concerned I thought it’d be interesting to put together a football transfers graph to see whether there are any interesting insights to be had.

It took me a while to find an appropriate source but I eventually came across transfermarkt.co.uk which contains transfers going back at least as far as the start of the Premier League in 1992.

I wrote a quick Python script to create a CSV file of all the transfers. This is what the file looks like:

$ head -n 10 data/transfers.csv
player,from_team,from_team_id,to_team,to_team_id,fee,season
Martin Keown,Everton,29,Arsenal FC,11,"2,10 Mill. £",1992-1993
John Jensen,Bröndby IF,206,Arsenal FC,11,"1,12 Mill. £",1992-1993
Alan Miller,Birmingham,337,Arsenal FC,11,,1992-1993
Jim Will,Sheffield Utd.,350,Arsenal FC,11,,1992-1993
David Rocastle,Arsenal FC,11,Leeds,399,"1,68 Mill. £",1992-1993
Perry Groves,Arsenal FC,11,Southampton FC,180,595 Th. £,1992-1993
Ty Gooden,Arsenal FC,11,Wycombe Wand.,2805,?,1992-1993
Geraint Williams,Derby,22,Ipswich Town,677,525 Th. £,1992-1993
Jason Winters,Chelsea U21,9250,Ipswich Town,677,?,1992-1993

I’m going to create the following graph and then we’ll write some queries which explore chains of transfers involving players and clubs.

2015 07 15 07 28 11

I wrote a few import scripts using Neo4j’s LOAD CSV command, having set up the appropriate indexes first:

create index on :Team(id);
create index on :Season(name);
create index on :Transfer(description);
create index on :Player(name);
// teams
load csv with headers from "file:///Users/markneedham/projects/football-transfers/data/teams.csv" as row
merge (team:Team {id: toint(row.team_id)})
on create set team.name = row.team;
 
// seasons
load csv with headers from "file:///Users/markneedham/projects/football-transfers/data/transfers.csv" as row
merge (season:Season {name: row.season})
ON CREATE SET season.starts =  toint(split(season.name, "-")[0]);
 
// players
load csv with headers from "file:///Users/markneedham/projects/football-transfers/data/transfers.csv" as row
merge (player:Player {name: row.player});
 
// transfers
load csv with headers from "file:///Users/markneedham/projects/football-transfers/data/transfers.csv" as row
match (from:Team {id: toint(row.from_team_id)})
match (to:Team {id: toint(row.to_team_id)})
match (season:Season {name: row.season})
match (player:Player {name: row.player})
 
merge (transfer:Transfer {description: row.player + " from " + from.name + " to " + to.name})
merge (transfer)-[:FROM_TEAM]->(from)
merge (transfer)-[:TO_TEAM]->(to)
merge (transfer)-[:IN_SEASON]->(season)
merge (transfer)-[:PLAYER]->(player);
 
// connect transfers
match (season)<-[:IN_SEASON]-(transfer:Transfer)-[:PLAYER]->(player)
WITH player, season, transfer
ORDER BY player.name, season.starts
WITH player, COLLECT({s: season, t: transfer}) AS transfers
UNWIND range(0, length(transfers)-2) AS idx
WITH player, transfers[idx] AS t1, transfers[idx +1] AS t2
WITH player, t1.t AS t1, t2.t AS t2
MERGE (t1)-[:NEXT]->(t2);

All the files and scripts are on this gist if you want to play around with the data. The only thing you’ll need to change is the file path on each of the ‘LOAD CSV’ lines.

The ‘connect transfers’ query is a bit more complicated than the others – in that one we’re first ordering the transfers in ascending order grouped by player and then creating a linked list of a player’s transfers.

Now that we’ve got the data loaded let’s find out which player was transferred the most:

match path = (:Transfer)-[:NEXT*0..]->(transfer:Transfer)
where NOT (transfer)-[:NEXT]->()
RETURN path 
ORDER BY LENGTH(path) DESC
LIMIT 1
Graph  22

Which other players have moved teams frequently?

match path = (first:Transfer)-[:NEXT*0..]->(transfer:Transfer),
             (player)<-[:PLAYER]-(transfer)
where NOT ((transfer)-[:NEXT]->()) AND NOT ((first)<-[:NEXT]-())
RETURN player.name, LENGTH(path) AS numberOfTransfers 
ORDER BY numberOfTransfers DESC
LIMIT 10
 
==> +--------------------------------------+
==> | player.name      | numberOfTransfers |
==> +--------------------------------------+
==> | "Craig Bellamy"  | 7                 |
==> | "David Unsworth" | 6                 |
==> | "Andrew Cole"    | 6                 |
==> | "Peter Crouch"   | 6                 |
==> | "Les Ferdinand"  | 5                 |
==> | "Kevin Phillips" | 5                 |
==> | "Mark Hughes"    | 5                 |
==> | "Tommy Wright"   | 4                 |
==> | "Carl Tiler"     | 4                 |
==> | "Don Hutchison"  | 4                 |
==> +--------------------------------------+
==> 10 rows

What are the most frequent combinations of clubs involved in transfers?

match (from)<-[:FROM_TEAM]-(t:Transfer)-[:TO_TEAM]->(to), (t)-[:PLAYER]->(p)
RETURN from.name, to.name, COUNT(*) AS times, COLLECT(p.name) AS players
ORDER BY times DESC
LIMIT 10
 
==> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | from.name           | to.name               | times | players                                                                                                                                                                                                    |
==> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | "West Ham United"   | "Queens Park Rangers" | 13    | ["Keith Rowland","Iain Dowie","Tim Breacker","Ludek Miklosko","Bertie Brayley","Terrell Forbes","Steve Lomas","Hogan Ephraim","Nigel Quashie","Danny Gabbidon","Kieron Dyer","Robert Green","Gary O'Neil"] |
==> | "Tottenham Hotspur" | "Portsmouth FC"       | 12    | ["Paul Walsh","Andy Turner","Rory Allen","Justin Edinburgh","Tim Sherwood","Teddy Sheringham","Noé Pamarot","Pedro Mendes","Sean Davis","Jermain Defoe","Younès Kaboul","Kevin-Prince Boateng"]            |
==> | "Liverpool FC"      | "West Ham United"     | 12    | ["Julian Dicks","David Burrows","Mike Marsh","Don Hutchison","Neil Ruddock","Titi Camara","Rob Jones","Rigobert Song","Craig Bellamy","Joe Cole","Andy Carroll","Stewart Downing"]                         |
==> | "Manchester United" | "Everton FC"          | 9     | ["Andrey Kanchelskis","John O'Kane","Jesper Blomqvist","Phil Neville","Tim Howard","Louis Saha","Darron Gibson","Sam Byrne","Tom Cleverley"]                                                               |
==> | "Newcastle United"  | "West Ham United"     | 9     | ["Paul Kitson","Shaka Hislop","Stuart Pearce","Wayne Quinn","Lee Bowyer","Kieron Dyer","Scott Parker","Nolberto Solano","Kevin Nolan"]                                                                     |
==> | "Blackburn Rovers"  | "Leicester City"      | 9     | ["Steve Agnew","Tim Flowers","Callum Davidson","John Curtis","Keith Gillespie","Craig Hignett","Nils-Eric Johansson","Bruno Berner","Paul Gallagher"]                                                      |
==> | "Chelsea FC"        | "Southampton FC"      | 8     | ["Ken Monkou","Kerry Dixon","Neil Shipperley","Mark Hughes","Paul Hughes","Graeme Le Saux","Jack Cork","Ryan Bertrand"]                                                                                    |
==> | "Birmingham City"   | "Coventry City"       | 8     | ["David Rennie","John Gayle","Liam Daish","Gary Breen","Stern John","Julian Gray","Lee Carsley","Gary McSheffrey"]                                                                                         |
==> | "Southampton FC"    | "Fulham FC"           | 8     | ["Micky Adams","Kevin Moore","Terry Hurlock","Maik Taylor","Alan Neilson","Luís Boa Morte","Antti Niemi","Chris Baird"]                                                                                    |
==> | "Portsmouth FC"     | "Stoke City"          | 8     | ["Kevin Harper","Lewis Buxton","Anthony Pulis","Vincent Péricard","Asmir Begovic","Marc Wilson","Elliot Wheeler","Alex Grant"]                                                                             |
==> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> 10 rows

Are there ever situations where players get transferred in both directions?

match (from)<-[:FROM_TEAM]-(t:Transfer)-[:TO_TEAM]->(to), (t)-[:PLAYER]->(player)
where id(from) < id(to)
WITH from, to, COUNT(*) AS times, COLLECT(player.name) AS players
match (to)<-[:FROM_TEAM]-(t:Transfer)-[:TO_TEAM]->(from), (t)-[:PLAYER]->(player)
RETURN from.name, to.name, times, COUNT(*) as otherWayTimes, players, COLLECT(player.name) AS otherWayPlayers
ORDER BY times + otherWayTimes DESC
LIMIT 10
 
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | from.name           | to.name               | times | otherWayTimes | players                                                                                                                                                                                                    | otherWayPlayers                                                                                                                                                                    |
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | "Tottenham Hotspur" | "Portsmouth FC"       | 12    | 5             | ["Paul Walsh","Andy Turner","Rory Allen","Justin Edinburgh","Tim Sherwood","Teddy Sheringham","Noé Pamarot","Pedro Mendes","Sean Davis","Jermain Defoe","Younès Kaboul","Kevin-Prince Boateng"]            | ["Jermain Defoe","Niko Kranjcar","Younès Kaboul","Peter Crouch","Darren Anderton"]                                                                                                 |
==> | "West Ham United"   | "Liverpool FC"        | 4     | 12            | ["Julian Dicks","Daniel Sjölund","Yossi Benayoun","Javier Mascherano"]                                                                                                                                     | ["Stewart Downing","Andy Carroll","Joe Cole","Craig Bellamy","Rigobert Song","Titi Camara","Rob Jones","Neil Ruddock","Don Hutchison","Julian Dicks","Mike Marsh","David Burrows"] |
==> | "West Ham United"   | "Queens Park Rangers" | 13    | 2             | ["Keith Rowland","Iain Dowie","Tim Breacker","Ludek Miklosko","Bertie Brayley","Terrell Forbes","Steve Lomas","Hogan Ephraim","Nigel Quashie","Danny Gabbidon","Kieron Dyer","Robert Green","Gary O'Neil"] | ["Andy Impey","Trevor Sinclair"]                                                                                                                                                   |
==> | "West Ham United"   | "Tottenham Hotspur"   | 5     | 8             | ["Jermain Defoe","Frédéric Kanouté","Michael Carrick","Jimmy Walker","Scott Parker"]                                                                                                                       | ["Sergiy Rebrov","Mauricio Taricco","Calum Davenport","Les Ferdinand","Matthew Etherington","Bobby Zamora","Ilie Dumitrescu","Mark Robson"]                                        |
==> | "West Ham United"   | "Portsmouth FC"       | 8     | 5             | ["Martin Allen","Adrian Whitbread","Marc Keller","Svetoslav Todorov","Hayden Foxe","Shaka Hislop","Sébastien Schemmel","Hayden Mullins"]                                                                   | ["Stephen Henderson","Teddy Sheringham","Shaka Hislop","Marc Keller","Lee Chapman"]                                                                                                |
==> | "Newcastle United"  | "West Ham United"     | 9     | 3             | ["Paul Kitson","Shaka Hislop","Stuart Pearce","Wayne Quinn","Lee Bowyer","Kieron Dyer","Scott Parker","Nolberto Solano","Kevin Nolan"]                                                                     | ["Demba Ba","Lee Bowyer","David Terrier"]                                                                                                                                          |
==> | "Birmingham City"   | "Coventry City"       | 8     | 4             | ["David Rennie","John Gayle","Liam Daish","Gary Breen","Stern John","Julian Gray","Lee Carsley","Gary McSheffrey"]                                                                                         | ["Scott Dann","David Burrows","Peter Ndlovu","David Smith"]                                                                                                                        |
==> | "Manchester City"   | "Portsmouth FC"       | 8     | 4             | ["Paul Walsh","Carl Griffiths","Fitzroy Simpson","Eyal Berkovic","David James","Andrew Cole","Sylvain Distin","Tal Ben Haim"]                                                                              | ["Benjani","Gerry Creaney","Kit Symons","Paul Walsh"]                                                                                                                              |
==> | "Blackburn Rovers"  | "Southampton FC"      | 5     | 6             | ["David Speedie","Stuart Ripley","James Beattie","Kevin Davies","Zak Jones"]                                                                                                                               | ["Zak Jones","Egil Östenstad","Kevin Davies","Alan Shearer","Jeff Kenna","Tim Flowers"]                                                                                            |
==> | "AFC Bournemouth"   | "West Ham United"     | 3     | 8             | ["Keith Rowland","Paul Mitchell","Scott Mean"]                                                                                                                                                             | ["Steve Jones","Matt Holland","Mohammed Berthé","Scott Mean","Paul Mitchell","Jamie Victory","Mark Watson","Stephen Purches"]                                                      |
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Any players who go back to the same club they were at previously?

match (player:Player)<-[:PLAYER]-(t1:Transfer)-[:FROM_TEAM]->(from)<-[:TO_TEAM]-(t2:Transfer)-[:FROM_TEAM]->(to),
      (t2)-[:PLAYER]->(player), (t1)-[:TO_TEAM]->(to)
WHERE ID(to) < ID(from)
WITH player, COLLECT([ from.name, " ⥄ ", to.name]) AS teams
RETURN player.name, 
       REDUCE(acc = [], item in teams | acc  + REDUCE(acc2 = "", i in item | acc2 + i)) AS thereAndBack
ORDER BY LENGTH(thereAndBack) DESC
LIMIT 10
 
==> +-------------------------------------------------------------------------------------+
==> | player.name       | thereAndBack                                                    |
==> +-------------------------------------------------------------------------------------+
==> | "Mark Stein"      | ["Stoke City ⥄ Chelsea FC","Ipswich Town ⥄ Chelsea FC"]         |
==> | "Peter Beagrie"   | ["Bradford City ⥄ Everton FC","Bradford City ⥄ Wigan Athletic"] |
==> | "Richard Dryden"  | ["Southampton FC ⥄ Stoke City","Southampton FC ⥄ Swindon Town"] |
==> | "Robbie Elliott"  | ["Bolton Wanderers ⥄ Newcastle United"]                         |
==> | "Elliot Grandin"  | ["Blackpool FC ⥄ Crystal Palace"]                               |
==> | "Robert Fleck"    | ["Chelsea FC ⥄ Norwich City"]                                   |
==> | "Paul Walsh"      | ["Portsmouth FC ⥄ Manchester City"]                             |
==> | "Rick Holden"     | ["Manchester City ⥄ Oldham Athletic"]                           |
==> | "Gary McAllister" | ["Liverpool FC ⥄ Coventry City"]                                |
==> | "Lee Bowyer"      | ["West Ham United ⥄ Newcastle United"]                          |
==> +-------------------------------------------------------------------------------------+

That’s all I’ve got for now – if you can think of any other interesting avenues to explore let me know and I’ll take a look.

Categories: Blogs

Pitfall of Scrum: Problem-Solving in the Daily Scrum

Learn more about our Scrum and Agile training sessions on WorldMindware.com

The Daily Scrum should not be used to find solutions to problems (obstacles, impediments) raised. Instead, keep the meeting very short and have those problem-solving conversations afterwards with only those who are interested. The ScrumMaster facilitates this meeting to keep it on track. The Daily Scrum is timeboxed to a maximum of 15 minutes, but often should be even less. With a good physical task board, a Daily Scrum can often be done in less than a minute simply by each team member pointing at the pieces of work they are working on.

From the Scrum Guide:

The Development Team or team members often meet immediately after the Daily Scrum for detailed discussions, or to adapt, or replan, the rest of the Sprint’s work.

In other words, don’t have those discussions during the Daily Scrum! The Daily Scrum is essential to creating transparency and implementing the Scrum value of Openness. The three questions of the Daily Scrum are effectively:

  1. What did I do since the last time we checked in as a team?
  2. What am I planning to do before the next check in time?
  3. What impediments, if any, are preventing us from getting our work done?

Each member of the team takes a turn and answers those three questions. This doesn’t have to be completely stilted, but it should be Focused (another value of Scrum) and efficient so that the need for other meetings is minimized. Accomplishing this takes some practice. The ScrumMaster helps the team to keep the timebox, but at first, a team might have challenges with this.

Struggling with the Daily Scrum

There are a some common reasons that a team might struggle with wanting to problem solve in the Daily Scrum:

  • One team member doesn’t know what to do next and it devolves into re-planning right there and then. A quick suggestion or two is probably fine, but it is a very steep slippery slope. A team can easily get into the habit of always doing this! The ScrumMaster needs to be vigilant about recommending that the discussion be taken up after the Daily Scrum is concluded in order to avoid this pitfall. This suggestion will be common when a team is first starting out.
  • One person mentions an impediment that someone else knows how to solve… and a third person has a different idea of solving it. In this situation it is much better for interested team members to just simply indicate “I have an idea for that,” and let the Daily Scrum continue. Then after the Daily Scrum those people have a quick discussion. This avoids wasting the time of everyone on the team with something that is only interesting to a few.
  • An individual doesn’t seem to have anything to report and other team members try to elicit more information. This should really be something that the ScrumMaster or the team’s coach should take up with the individual. It may be that there is an impediment that the person is uncomfortable sharing openly with the whole team. There is a subtle pitfall that may be revealed here: that the team does not have the safety to self-organize.
  • Disagreement about what to do next. This type of problem is the hardest to deal with because many people will feel that disagreements need to be resolved before any action can be taken. A good ScrumMaster will actually encourage competing ideas to be attempted. Learn by doing instead of by argument and analysis. This is the fundamental shift in culture that Scrum is attempting to put in place: an empirical approach to work rather than a defined approach.

Just beware: yet another pitfall (although not common) is to decide that the Daily Scrum shouldn’t be daily because it is taking so long. Unfortunately, making this change will often just make the meetings even longer until they devolve back into weekly status meetings reporting to the team lead!!! Remember that it’s not Scrum anymore if your team doesn’t meet together daily.

Ultimately, if a team is struggling with the Daily Scrum in any way, this is a valid topic for discussion in the Sprint Retrospective.

This article is a follow-up article to the 24 Common Scrum Pitfalls written back in 2011.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to informationPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post Pitfall of Scrum: Problem-Solving in the Daily Scrum appeared first on Agile Advice.

Categories: Blogs

Python: UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xe2 in position 0: ordinal not in range(128)

Mark Needham - Wed, 07/15/2015 - 08:20

I was recently doing some text scrubbing and had difficulty working out how to remove the ‘†’ character from strings.

e.g. I had a string like this:

>>> u'foo †'
u'foo \u2020'

I wanted to get rid of the ‘†’ character and then strip any trailing spaces so I’d end up with the string ‘foo’. I tried to do this in one call to ‘replace':

>>> u'foo †'.replace(" †", "")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

It took me a while to work out that “† ” was being treated as ASCII rather than UTF-8. Let’s fix that:

>>> u'foo †'.replace(u' †', "")
u'foo'

I think the following call to unicode, which I’ve written about before, is equivalent:

>>> u'foo †'.replace(unicode(' †', "utf-8"), "")
u'foo'

Now back to the scrubbing!

Categories: Blogs

New Webinar: DevOps, SAFe and the Path to Better Agile Development

Agile Product Owner - Tue, 07/14/2015 - 18:53

A Perspective on DevOps from SAFe, IBM and Telstra

Hi everyone,

Alex just participated in a webinar hosted by IBM that provides SAFe, IBM and customer implementation perspectives on DevOps. Other participants include:

  • Tony Christensen, GM/Agile Competency Lead, Telstra
  • Amy Silberbauer, Executive IT & Solution Specialist, SAFe and DevOps

The session was facilitated by IBM’s Kelli Houston. The customer is Telstra, Australia’s largest telecommunications provider, who has recently implemented SAFe, with a rollout that includes over a thousand software development practitioners. That’s agile at scale!

It’s an interesting construct—a distributed panel discussion combining these unique perspectives. While the focus is on DevOps, there are some interesting findings around some “more obvious” constructs like the value of the common taxonomy that SAFe provides, the role of leadership in facilitating flow at the program and portfolio levels, the value and necessity of training to the new way of working, the cultural transformation that DevOps really entails, and more. This 30+ minute webinar provides a lot of value in understanding how to prevent agile from simply creating more un-deployed code laid on the doorsteps of the IT Ops team.

You can watch it here:

https://www.youtube.com/watch?v=OTS3v0b3m_c

Stay SAFe!

–Dean

 

 

Categories: Blogs

Estimating Placeholder Stories is a Bad Practice

Leading Agile - Mike Cottmeyer - Tue, 07/14/2015 - 13:59

Placeholder stories, in general, are a bad idea. Estimating placeholder stories to reserve capacity or to get credit is a very bad idea.

Define “Placeholder Stories”

Of course, all stories are “placeholders for a conversation”, but that’s not what I’m talking about here.

I am also not talking about things like “Refactor the such-n-such class as we start work on the something-or-another Epic”. I’m not talking about having a known customer problem but not yet knowing what to do about it, if anything. Those are specifically identifiable work. Those should be in the backlog (for the sprint or the release), and I don’t call them placeholders.

I’m not wild about having time-tracking stories in the sprint or release backlog because they need to be backed out of certain metrics such as number of stories completed (throughput). But I understand why organizations have them. Let’s call those time-tracking stories instead of placeholders.

By “placeholder stories” what I mean are stories for known-unknowns (or even unknown-unknowns) such as:

– we know we’ll have some production support work to do… 2 points

– we might have to do some sales support… 1 point

– we know we’ll have to do deal with some high priority defects that come up during the sprint… 3 points

– we could get interrupted for some high priority expedite work… reserve 8 points

– the build server is flakey and it might break again… 2 points

– “placeholder story for other unknown work that we might have to do”… 3 points

– end of release regression testing… 60 points

My main complaint is estimating them.

Visibility is Good

I’ve seen teams put a placeholder story in each sprint just as a place to keep record of their customer calls (mainly to get credit). They’ll add a line in the description or a (sub-)task for each new support call. I’m not completely opposed to the placeholder story itself in this case because it’s really just another type of time-tracking story. Also, making the work visible with sub-tasks is a fine idea. But estimating it is wonky.

Negatively Impacts Release Planning

Velocity is a useful input to sprint planing, but it is far more important and useful for release planning. If velocity includes defects and other placeholders, release planning actually becomes more difficult.

Keeping up with how much of the velocity the PO could bank on and how much will go to defects and other placeholders gets tiresome and is error prone.

When the Product Owner is considering the cost or duration of a batch of stories (for example, a new feature or an epic), that PO can’t just do simple math (backlog size / velocity). They have to ask someone to put placeholder stories for support work into the epic or they have to ask someone to tell them what’s the velocity of just the real user stories.

Placeholders make it harder to do what-if scenarios for the release: “What if we trimmed some scope and brought the release date in 2 months? What if we do this expedite and extend the release 1 month?” We have to consider where the placeholders fall in the backlog and scale them up or down. Or if there is a placeholder slotted in advance for every future sprint, then we have lots of placeholders polluting our backlog.

Estimating placeholders causes your velocity to be inflated relative to the backlog of known work for the release. Not estimating placeholders for known-unknowns will simplify release planning.

Therefore, don’t put placeholder junk in your velocity. If you use velocity or throughput for release planning purposes, and you probably should if you are making any release commitments at all, then your velocity metric should only count value stories — items in the backlog that the Product Owner or Customer wants, items from your story map and your Epic/Feature/Story decomposition.

Not Needed for Contingency

Some people use estimated placeholders for buffering in a release plan. They have to do this because they put points on everything, on all the unplanned stuff too. Don’t estimate that stuff. The unplanned and unknown stuff should serve to lower your velocity.

Track the historical rate of production support separately from your velocity. Track defect rates too. Understand the trend and distribution pattern across a release. Then take that into consideration when putting the release plan together. For example, that information helps me decide what to use for my planning velocity, how many or which sprints to include in the average and whether to adjust it further. It also helps me decide how many sprints of active development to plan for. If you take a week or a couple sprints at the end of your release to do regression testing, just plan for that in your schedule. You don’t need to put points on it.

Moving placeholders around as release dates change and adjusting them as the buffer is consumed gets tiresome. Worse, it’s a complication that creates unnecessary opportunities for error.

Plan releases conservatively: don’t include known-unknowns in your velocity; but do include known-unknowns in your release plan.

Not Needed for Sprint Planning

Usually when teams use placeholders it’s because they want to estimate story points for that work in order to reserve some capacity in the sprint. Putting a placeholder in every sprint and calling it 2 points to reserve capacity is unnecessary.

Plan sprints based on the amount of planned points of stuff the Product Owner wanted that you got done in the last sprint. If you use (sub-)tasks with hours estimates and an hours-based burndown, then reduce your capacity for these unknown things, but base it on how many planned hours for known-knowns you were able to do in prior sprints.

If your build server is flakey or if you need to do some maintenance on your VM server, either decide to fix it or don’t. Don’t put a placeholder in your sprint just in case. That’s poor planning.

Isn’t Relative Estimation

Relative estimation works when comparing like kinds of work — known-knowns and actual user stories. You don’t know how many production support issues you are going to have or how long it’s going to take to resolve them. Estimating that stuff is much less accurate than estimating known-knowns. General production support and other overhead shouldn’t show up as estimated items in your sprint or release backlog. Just let support activity be a drag on your velocity.

What About Defects and Expedites?

Defects should show up in your backlog once there is a specific issue that needs fixing — a known-known. Same for high priority expedite work. Add it to your sprint once it becomes a known-known.

But don’t estimate defects. Just let them be a drag on your velocity. Track the historical rate of defects separately. You can reserve capacity in your sprint and release plan without putting points on a placeholder for defects that haven’t been discovered yet.

Getting Credit for Work Done

If your team is using and estimating placeholders to get credit, Stop that. It’s a dysfunction in the organization if people feel the need to inflate their velocity or get credit.

Conclusion

There is no good reason to put points on placeholder stories. Keep unplanned stuff out of your velocity number. This leads to simpler and more conservative release planning.

The post Estimating Placeholder Stories is a Bad Practice appeared first on LeadingAgile.

Categories: Blogs

Dipping Toes in Other Development Communities

My dad, even with a serious head injury that ended his working life, regularly attended a computer club where he could chat with a community of fellow geeks who accepted him head injury and all. I was always glad he had somewhere he could be accepted, but I never realized I’d end up attending and helping to run programming user groups in my own career.

I first went to a Java Users Group in Sacramento, SacJUG. I had been getting deeper into Java and figured it was time to see what the local community was like. I found a home with some fellow geeks and I attended regularly for the next ten years or so. I even eventually hired multiple developers I met at SacJUG and I appreciated being able to geek out on the language and share war stories. A few years later I got deeply into Ruby in my spare time with the advent of Rails and eventually helped setup the Sacramento Ruby Meetup. It would be a few more years until I got paid to do Ruby, but met some great developers along the way and I still regularly attend. Only a year or so later I helped found the Sacramento Groovy User’s group which continues today as essentially a JVM languages group.

All this experience with user groups has led me to experiment with visiting other user groups from time to time. A few months ago I showed up for an Angular group and met a lot of front-end specific developers I don’t mix with regularly.

If you haven’t tried out attending a user group I encourage you to try it. It only costs you an hour or two and the benefits are worth it. Things like:

  • Seeing the size of say the node.js community in your town and getting a sense of how a new language or toolset is catching on.
  • Getting exposure to something new with a group of programmers.
  • Meeting fellow developers who are trying to come up to speed or stay on top of technologies.
  • Finding a great new candidate for your shop. The developers who regularly attend user groups tend to be more motivated and engaged employees and you can get a pretty good sense of their skills just from chatting.
  • Getting out of the house since many of us are introverts. With the shared context it’s much easier than say the annual holiday party.
  • Practice speaking if you work up the nerve in a low stress atmosphere.
  • A sense of whether a particular language/community is on the rise or fall.
Categories: Blogs

Scrum Mythbusters

TV Agile - Mon, 07/13/2015 - 18:52
As Scrum popularity continues to grow, so do associated myths and misunderstandings. This session will debunk many of these common misinterpretations of the world’s most popular Agile Framework. Some of the myths that will be debunked during this session include: * Scrum can’t work for production support * Scrum can’t work for teams practicing continuous […]
Categories: Blogs