Skip to content

Blogs

12 years, 12 lessons working at ThoughtWorks

thekua.com@work - Fri, 04/01/2016 - 15:15

I’ve been at ThoughtWorks for 12 years. Who would have imagined? Instead of writing about my reflections on the past year, I thought I would do something different and post twelve key learnings and observations looking back over my career. I have chosen twelve, not because there are only twelve, but because it fits well with the theme of twelve years.

1. Tools don’t replace thinking

In my years of consulting and working with many organisations and managers I have seen a common approach to fixing problems, where a manager believes a tool will “solve” the given problem. This can be successful where a problem area is very well understood, unlikely to have many exceptions and everyone acts in the same manner. Unfortunately this doesn’t reflect many real-world problems.

Too many times I have witnessed managers implement an organisational-wide tool that is locked down to a specific way of working. The tool fails to solve the problem, and actually blocks real work from getting done. Tools should be there to aid, to help prevent known errors and to help us remember repeated tasks, not to replace thinking.

2. Agile “transformations” rarely work unless the management group understand its values

Many managers make the mistake that only the people close to the work need to “adopt agile” when other parts of the organisation need to change at the same time. Co-ordinating this in enterprises takes a lot of time and skill with a focus on synchronising change at different levels of the organisation.

Organisations who adopt agile in only one part of their organisation face a real threat. As the old saying goes, “Change your organisation, or change your organisation.”

3. Safety is required for learning

Learning necessitates the making of mistakes. In the Dreyfus model, this means that particularly people in an Advanced Beginner stage, need to make mistakes in order to learn. People won’t risk making mistakes if they feel they will do a bad job, lose respect from their colleagues or potentially hurt other people in that process.

As a person passionate about teaching and learning, I find ways to create a safe space for people to fail, and in doing so, make the essential mistakes they need to properly learn.

4. Everyone can be a leader

I have written about this topic before, but it is such an important observation. I see a common mental model trap where people feel the need to be given the role of a leader, in order to act like a leader. People can demonstrate acts of leadership regardless of their title and can do so in many different ways, simply by taking action on something without the explicit expectation or request for it.

5. Architects make the best decisions when they code

In the Tech Lead courses I run, I advocate for Tech Leads to spend at least 30% of their time coding. Spending time with the code helps build trust, respect and a current understanding of the system. Making architectural decisions without regard for the constraints of the current system are often bad decisions.

6. Courage is required for change

I miss people talking about the XP values, one of which includes Courage. Courage is required for acts of leadership, taking on the risk to fail and the risk/reward of attempting something new. Where there is no risk, there is often little reward.

7. Congruence is essential for building trust

Beware of the old age maxim, “Do as I say, not as a I do.” In reality, regardless of what you say, people will remember how you act, first and foremost. Acting congruently is making sure that your actions follow your words. Acting incongruently destroys trust. Saying “no” or “not now” is better than promising to do something by a certain time, only to not deliver it.

8. Successful pair programming correlates with good collaboration

Although not all pair programming environments are healthy, I do believe that when it works well, teams tend to have better collaborative cultures. Many developers prefer the anti-pattern of (long lived) branch-based development because it defers feedback and sources of potential conflict.

I consider (navigable) conflict a healthy sign of collaborative teams. Deferring feedback, such as is the case with code reviews on long-lived branches tends to lead to more resentment because it is delivered so late.

9. Multi model thinking leads to more powerful outcomes

One of my favourite subjects at University, was Introduction to Philosophy where we spent each week in the semester studying a different philosopher. Over the course of my career, I have come to appreciate the value of diversity, and to see a problem through multiple lenses. Systems thinking also recognises that facts can be interpreted in different ways, leading to newer ideas or solutions which may be combined for greater effect.

10. Appreciate that everyone has different strengths

Everyone is unique, each with their own set of strengths and weaknesses. Although we tend to seek like-minded people, teams are better off with a broader set of strengths. A strength in one area may be a weakness in a certain context, and teams are stronger when they have a broader set of strengths. Differences in strengths can lead to conflict but healthy teams appreciate the differences that people bring, rather than resent people for them.

11. Learning is a lifelong skill

The world constantly changes around us and there are always opportunities to learn some new skill, technique or tool. We can even learn to get better at learning and there are many books like Apprenticeship Patterns and The First 20 Hours which can give you techniques to get better at this.

12. Happiness occurs through positive impact

The well known book, Drive, talks about how people develop happiness through working towards a certain purpose. In my experience, this is often about helping people find ways to have a positive impact on others, which is why our Pillar 2 (Champion software excellence and revolutionize the IT industry) and Pillar 3 (Advocate passionately for social and economic justice) values are really important for us.

Conclusion

The twelve points above are not the only lessons I have learned in my time at ThoughtWorks but they are some of the more key learnings that help me help our clients.

Categories: Blogs

Testing In Ansible

This is a topic I brushed up against yesterday and meant to blog about it at the end of the day but got a little busy. A lot of times when provisioning boxes locally in vagrant I’ve thought it would be incredibly useful to be able to automatically test the system to ensure all the expected bits are provisioned as expected.

I’ll probably throw together a nice public demo but the short and skinny is to include a final ansible provisioning step after the normal step that runs a test playbook of sorts against the system. For us we just dumped our test tags into our main roles and tag them as test. Then in vagrant we exclude test tagged tasks and then in the test phase we only run those tagged tasks. Below is an example for one of our services to test that two service processes are running and that the load balancer is also serving up responses that are the same as those running on the two processes.

I’ve also heard of other tools in this space like ServerSpec which may fit your bill if you’re not running ansible or are running some mixed environment. So far I think ansible fits well here but you’re definitely going to be a little limited due to the tests being in yaml. Although you could hypothetically write some custom modules or resort to shell wizardry if you need something more advanced.

I’m really excited about this… the idea we could have full test suites with each of our ansible roles that can verify a whole swath of aspects like expected ulimits and the like is GREAT.

Categories: Blogs

Friday Functions: AWS ZSH Helper

This morning I’m going to go with a new recurring weekly post: Friday Functions! While some of it will aim to share my large inventory of zsh functions I’ve acquired over the years I’ll also be finding new additions if I run out of material. So it also serves to help me learn more!

This week’s function is probably only useful if you’re into AWS and use the awscli tool to interact with it from the command line. Using the awscli command direction can be quite verbose so some nice shortcuts are useful. I actually learned of this handy function from Kris’s awesome collection of zsh configuration and made a few small adaptions to it.

This is pretty useful. If you want to find all instances with http in the name you just run aws-instances-describe http.

Screen Shot 2016-03-31 at 6.14.42 PM

Or if you want to look for instances by a specific tag you can use the `-t` switch. For example, to find all instances with the worker_email role tag we can just run aws-instance-describe -t role worker_email. You can add -s to changed the filter to include the running state and like the actual call you can include multiple instances. So if you wanted to find all stopped instances with the taskhistory role you’d run aws-instance-describe -t role taskhistory -s stopped. The function sets this to default to running instances only since that’s what I’m looking for 99% of the time… looking for stopped or terminated instances is definitely the exception.

Hope this was interesting enough. Ideas, thoughts, comments or criticism are all welcome in the comments below! Let me know what you think! 🙂

Categories: Blogs

What Are Developers Really Paid To Do?

Derick Bailey - new ThoughtStream - Fri, 04/01/2016 - 13:30

It’s a question that most developers have a fast answer for: “WRITE CODE!” … but, is that really what you’re paid to do?

In this episode of Thoughts On Code I’ll explain why I don’t think your job is to just write code, after all.

Categories: Blogs

What's Keeping me Busy


I thought I’d take the opportunity to share some of things that keep me busy when I’m not doing my day job.

Podcasts

Note to Self - I recently started listening to this one and find it very engaging. The topics are around how technology fits into our lives. I find Manoush Zomorodi entertaining and informative.

Design Matters - This one isn’t really about design as much as it’s about how people got to where they are. Debbie Millman has been doing this podcast for over 10 years and each episode is an interview with someone somehow related to design. Examples include folks like Seth Godin, Daniel Pink and people more on design like Ayse Birsel (keep reading) and Doug Powell.

Freakanomics - This has been a favorite of mine for a long time. It takes an interesting look at topics such as the economic impact of not getting enough sleep. While there is always a look at the data, the podcast is very entertaining, hosted by Stephen J. Dubner, one of the authors of the book of the same title.

Clockwise - I’ve listened to a lot of tech podcasts and given up on them because they are too long. My threshold for any podcast is less than an hour, preferably half of that. Clockwise fits the bill. The two hosts have two guests and each of the four brings up a questions but the whole podcast is time boxed to 30 minutes.

Books

Design the Life you Love by Ayse Birsel. Ayse was one of the guests on Design Matters, where I heard about her book. This is actually a workbook. It uses the principles of Design Thinking and applies them to your life, as the title might imply.

David and Goliath - I’ve had this one on my bookshelf since last Christmas and finally dug into it. I’ve enjoyed everything I’ve read from Malcolm Gladwell and though I’m not very far into it, so far so good.

Running - I am going to run the Chicago Marathon this fall as part of Team RMHC, raising money for the Ronald McDonald House Charity. My wife and I volunteer at our local Ronald McDonald house and I think this is a great charity. You can read more about that on my other blog (and even donate to the cause)
Categories: Blogs

When Will Executives Step up to the Plate?

Learn more about transforming people, process and culture with the Real Agility Program

Hundreds of Canadian employees from corporations, businesses and organizations are attending training to become Certified ScrumMasters and Certified Product Owners under the aegis of the Agile umbrella. From testimonials received from almost all attendees, they are enthusiastic about this training. As many have written, the training is helping them think beyond the status quo, and they are excited!

They return to their workplaces, report to their managers, talk amongst themselves – and then what happens? Nothing. Nothing changes. Their learning, their positive motives to enact change, their hopes slowly dissipate in the face of ignorance and apathy.

Where’s the disconnect?

It seems the disconnect belongs to the executives. CEO’s, VP’s, upper management have been avoiding a work revolution happening right under their noses. The revolution began in 1998 with the creation of the Agile framework, resulting in the Agile Manifesto, http://agilemanifesto.org, written in February of 2001 by seventeen independent software practitioners.

Not only has Agile transformed software creation, but it has been proven to be of value for all areas of business enterprises and organizations beyond software and IT departments.

Are executives remaining willfully ignorant of a twenty-first century framework for creating more fulfilling workplaces and delivering greater value to their customers? Or will Executives learn what is happening at the grassroots and make changes to fulfill the hopes of employees?

This is a call to action. It is time for executives to step up to the plate.

Real testimonials about training can be found at http://www.worldmindware.com/CertifiedScrumMaster

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post When Will Executives Step up to the Plate? appeared first on Agile Advice.

Categories: Blogs

ScrumMaster or Armchair Psychologist? – Notes from a Webinar by Angela Johnson

Learn more about transforming people, process and culture with the Real Agility Program

Valerie Senyk

I like taking notes when I’m learning. I believe it helps me to retain what I’m hearing, although I have no solid proof to back that up.

March 9, 2016, I took advantage of a free webinar offered by the Scrum Alliance, with the above title. We were told 5,000 people were attending! Angela Johnson’s presentation was based on information from both the Scrum and psychology communities.

First, she reiterated many ideas that are commonly understood about the role of a ScrumMaster (SM): an internal coach, a servant leader, an active facilitator. The SM makes sure that the rules of Scrum are followed by a team, but focuses on interactions and outcomes. As in football, the SM is truly a coach.

She described how new SM’s often latch onto the mechanics of Scrum, but most important are the personal interactions of a team. When difficulties arise, it is important to ask: “Do we have a Scrum problem, or is it a people problem?”

Ms. Johnson cited two resources available to SM’s to understand people interactions and problems better. One is the previous publication by Dale Carnegie called “How to Win Friends and Influence People.” The other is a newer resource by Michael James available online as scrummasterchecklist.org. In it James covers ideas like “How is my team doing?…How is the product owner doing?…etc”

She also cited a fascinating book by Harvey Robbins and Michael Finley called “’The New’ Why Teams Don’t Work.” She spoke about the importance of goals and objectives for a team. Bad Teams have vague goals and objectives. Good Teams may have clearer goals and engage in barrier identification. But the Best Teams have clear, short-term goals with continuous high-priority goals and objectives in segments of 30 days or less; they also identify barriers to people and processes. Best Teams ultimately value differences among team members, and develop something she called “versatility plans in interpersonal relationships.” (I wanted to learn more about this idea and posed a question in the webinar…I’m still waiting for an answer.)

She then turned to psychology to discuss behavioral style differences in people. Four distinct personality types were explored: the Analytical (asks how?), the Driver (asks what?), the Amiable (asks who?) and the Expressive (asks why?). She believes a SM might help team members identify which personality quadrant they belong in so as to better understand each other. As well, in knowing the type of people who are in his/her team, a SM could adjust his/her communication and behavior to better reach each type.

This sounds like a great deal of work to me, but I think it would be an interesting exercise for a team to go through it at least once. The exercise itself, besides creating deeper understanding, could also lead to some “aha” moments and laughter.

Johnson went through a checklist of Harvey Robbins’ rules for building trust in a team or group of people, and I will list them here:

  • have clear, consistent goals

  • be open, fair, and willing to listen

  • be decisive (meet the definition of Done)

  • support all other team members

  • give credit to team members

  • be sensitive to the needs of members

  • respect others’ opinions

  • empower team members to act

  • adopt a “we” mentality

  • take responsibility for team actions

She then added tips about supporting versatility, such as: you can only create an environment that encourages self-motivation rather than motivate others directly; tell people how to interpret what you say, i.e. “What I’m about to say is to help…etc;” don’t overlook a variety of orientations, whether they are cultural, or gender-based. She advised that we need to be aware that orientation is very important to consider as the teams we work with have a greater number of people whose first language is not English.

She spoke about Scrum teams working within larger organizations. If the goal of Scrum is to produce greater value more quickly, then a SM should never have his/her attention split between more than one team. The SM has to be a teacher inside of an organization, to help management understand best practices – the SM is really the coach for a team, the Product Owner and the organization. Old habits die hard, so educating takes time. Don’t allow language to get in the way of this process. Don’t say: “That’s not Agile! That’s not Scrum! You’re doing it wrong!” Instead say, “When you say Agile, what do you mean?” or, “What is the problem we’re trying to solve?” A SM can always point out that we have a choice to work in the old way, or to try something new. We have an opportunity to improve the way we work.

Agile and Scrum, she emphasized, are not destinations – they are about continuous improvement.

This was a valuable webinar, and although it’s not possible to review everything that was in it, I hope this summarizes some of the important ideas Angela Johnson presented.

To hear the entire webinar, go to https://goto.webcasts.com/viewer/event.jsp?ei=1090551

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post ScrumMaster or Armchair Psychologist? – Notes from a Webinar by Angela Johnson appeared first on Agile Advice.

Categories: Blogs

Is Agile a Subset of Lean Manufacturing?

Leading Agile - Mike Cottmeyer - Thu, 03/31/2016 - 15:38

If you hang around agilists long enough, someone will mention lean manufacturing, Toyota Production System or Kanban. Since these concepts predate Agile, you might wonder how they relate, and perhaps why lean manufacturing wasn’t directly applied to software (until perhaps recently with Lean/Kanban). You might wonder whether Agile is just a subset of Lean.

Lean manufacturing floor

What is Lean Manufacturing?

Lean Manufacturing, sometimes called Just-in-Time manufacturing or Toyota Production System (TPS), originated in post-World War II Japan to cope with low cash, limited warehousing space, few natural resources and high unemployment. It emphasized driving waste out of the manufacturing cycle. Toyota’s definition of waste includes items we don’t normally consider—such as inventoried parts, “work in process,” variable work load—as well as the usual defects, product returns and overbuilding [Muda, Muri, Mura].

Lean manufacturers look very different from historic American assembly lines, which emphasized repetition, motion analysis and economies of scale. Toyota instead promoted greater autonomation, the philosophy that repetitive activities should be done by machines, while thinking and creative activities should be done by humans. So Toyota’s factory floors included more robots, employing humans in teams to assemble complex systems, analyze problems and devise solutions. Since American assembly lines exploited speed and employee utilization, Americans rarely stopped the assembly line. In contrast, Toyota workers ceremoniously pulled an andon (“paper lantern”) cord to announce discovery of a defect: conveyor belts stopped, managers made their way to the location of the pulled cord, and then workers and managers rapidly performed root cause problem analysis, devised a short-term fix and initiated a plan to permanently avoid the problem.

Lean Manufacturing fueled Toyota’s rise and dominance from almost nothing. It produced only about 2000 passenger vehicles between 1933 and 1947. By 1984, Toyota was producing about 4 million cars per year, virtually all in Japan.

In 1984, under pressure to produce cars in the USA and fearful that it did not understand American workers, Toyota partnered with General Motors to use lean manufacturing in GM’s shuttered Fremont factory, which had a history of antagonism, sabotage and drug use. Toyota wanted to experiment with “the real thing,” dismissing GM’s suggestion they find other workers, and rehired the motley crew. After Fremont employees were trained in TPS, the factory showed dramatic improvements in quality and profitability; it produced the highest quality cars GM offered [This American Life, NUMMI (26 March 2010)]. Other GM plants failed to apply the approach (which still stupefies me), and GM declared bankruptcy in 2010. Ironically, GM closed the Fremont plant for good the same year. As if Fremont is the location of some strange singularity, Tesla, today’s most cutting edge mass production car manufacturer, bought the Fremont plant in 2010 and now produces its cars there.

Since 2012, Toyota has produced more cars and more profit than any other car maker worldwide.

I study lean manufacturing and its many contributions to our society. Some will note how many agile practices arise from lean manufacturing. In fact, agilists even see the same stupefying management behavior, such as reversing agile practices despite overwhelming data showing Agile’s superiority in their own organizations. Agile stole freely from lean, and virtually all agilists acknowledge lean’s power. Agile and Lean Manufacturing are pals who share a lot of toys.

But is Agile a Subset of Lean?

Recently a colleague asked me whether Agile was a subset of Lean. Could lean manufacturing people produce an agile software transformation? I’m a “big tent” agile guy. The Scrum Alliance designates me as a Certified Enterprise Coach, but I’ve often spoken at Lean/Kanban conferences. Lean/Kanban folks include many “quants,” like me: people interested in data and statistics. Anyone who wants to produce better products for more beneficiaries is in my tent. So if reality makes Lean the big daddy of Agile, I’m cool with that.

But Agile and Lean Manufacturing (not to mention software development and manufacturing) are different. Someone armed solely with a lean manufacturing background is extremely unlikely to produce a successful agile transformation, unless they embrace agile practices that are outside lean and understand how Agile promotes creativity.

Agile and lean manufacturing both take action “just-in-time,” but they support different things. One supports innovation, the other supports optimization. Unfortunately, sometimes those goals conflict.

The article that led Jeff Sutherland and Ken Schwaber to invent Scrum, the dominant Agile methodology, is Hirotaka Takeuchi and Ikujiro Nonaka’s, “The New New Product Development Game,” Harvard Business Review (Jan 1986). This article compares product design to the fast-moving, barely-controlled game of rugby. It sparked a revolution, so most agilists know of it.

churchill-rugby-ireland-2390383-600x350

“Scrum to England,” Boocal.

Scrum and Agile practices focus on product development (i.e., design) rather than manufacturing. Scrum supports creativity, learning and innovation rather than analysis, waste reduction and control. For example, the article discusses the six characteristics of designing products in this new way:

  1. Built-in instability
  2. Self-organizing project teams
  3. Overlapping development phases
  4. “Multilearning”
  5. Subtle control
  6. Organizational transfer of learning

Here’s how Wikipedia describes lean manufacturing:

Lean manufacturing or lean production, often simply “lean”, is a systematic method for the elimination of waste (“Muda”) within a manufacturing system. Lean also takes into account waste created through overburden (“Muri”) and waste created through unevenness in work loads (“Mura”). Working from the perspective of the client who consumes a product or service, “value” is any action or process that a customer would be willing to pay for.

Essentially, lean is centered on making obvious what adds value by reducing everything else. Lean manufacturing is a management philosophy derived mostly from the Toyota Production System (TPS) (hence the term Toyotism is also prevalent) and identified as “lean” only in the 1990s. TPS is renowned for its focus on reduction of the original Toyota seven wastes to improve overall customer value, but there are varying perspectives on how this is best achieved. The steady growth of Toyota, from a small company to the world’s largest automaker, has focused attention on how it has achieved this success.

Note that key line, “lean centers on making obvious what adds value by reducing everything else.” Lean applied in isolation doesn’t generate innovative designs fast, because its focus is to remove variation, randomness and the unexpected. Creative activity, where we design and build something never built before—exhibited more or less in virtually all software development—introduces variation inherently. Creativity and unpredictability often go together; the more innovative a creative act is, the more variability it produces. When careless lean advocates measure and control variation without leaving room for natural variation in creativity, they may throw out the baby with the bathwater.

Agile and Lean practices can work well together, when their contributions are respected—waste is created in software development (technical debt, manual testing, component team dependencies, etc.) and lean manufacturing strategies can help. In fact, virtually all Scrum courses include material on Kanban, Theory of Constraints and A3, which come from lean manufacturing. If lean manufacturing strategies dominate, however,  creativity and innovation, key aspects of software development, can suffer.

Very recently, “Lean Startup” appeared on the scene (2003 “Four Steps to the Epiphany” by Steve Blank, and 2011 “Lean Startup” by Eric Reis). Lean Startup accelerates innovation by discarding the idea that we have to create something with customer value, and that instead we seek market learning value first. Lean Startup is radical, and its application has created modern, constantly-experimenting behemoths such as Facebook. By learning more about the market, we waste less effort creating products people won’t likely buy or use. One could argue that “Lean Startup” is just a “very big lean manufacturing,” and from that newly expanded vision I might have to agree. However, lean manufacturing texts did not include the waste of “not meet customer demand or specifications” until [Womack 2003]. because they focus on designs that are already largely completed. 

Some agilists push Lean/Kanban, which is an adaptation of lean manufacturing for IT, as a software development practice. Lean/Kanban emphasizes none of Takeuchi and Nonaka’s six characteristics. Despite having appeared 40 years before Agile, lean manufacturing has gained little hold in software development. However, Lean/Kanban appears to work well for IT operations (configuration and deployment) activities. I attribute this to the lower need for innovation in operations.

When colleagues advocate Lean/Kanban in software development, and especially if they are transitioning from Scrum, I always ask them to institute a metric-driven rhythmic retrospective (which I discuss here), and ask them to compare their Kanban results to their Scrum over time. Without a rhythmic retrospective, I have found Lean/Kanban to be difficult to sustain, and I want organizations to help more people with higher quality products for a long time. I’ve seen some sad unravelings of agility when people selectively adopt Lean/Kanban practices; but when people include metric-driven Retrospectives, it seems to last and improve more rapidly.

With historic data supporting Agile, managers take significant risk when they contemplate turning back the clock, even for something as agile-friendly as lean manufacturing. Waterfall projects fail at a rate of about 29% and Agile projects fail at 9%; waterfall succeeds 14% of the time and agile succeeds 44% of the time; the rest are “challenged projects” that partially achieved their goals and are being used (Standish Group Chaos Report). 

This is the third time I’ve read the Takeuchi and Nonaka paper. Every time I do I find a new subtlety or insight. I’m surprised how prescient it was. It’s an easy read, and I’d invite you to read it yourself.

You might ask yourself, “Is Lean a subset of Agile?” I sometimes wonder that myself.

There is nuance here, I really do apply lean manufacturing techniques daily in my Agile work, and I welcome alternative views. If you comment, I’ll happily respond.

The post Is Agile a Subset of Lean Manufacturing? appeared first on LeadingAgile.

Categories: Blogs

Is Agile a Subset of Lean Manufacturing?

Leading Agile - Mike Cottmeyer - Thu, 03/31/2016 - 15:38

If you hang around agilists long enough, someone will mention lean manufacturing, Toyota Production System or Kanban. Since these concepts predate Agile, you might wonder how they relate, and perhaps why lean manufacturing wasn’t directly applied to software (until perhaps recently with Lean/Kanban). You might wonder whether Agile is just a subset of Lean.

Lean manufacturing floor

What is Lean Manufacturing?

Lean Manufacturing, sometimes called Just-in-Time manufacturing or Toyota Production System (TPS), originated in post-World War II Japan to cope with low cash, limited warehousing space, few natural resources and high unemployment. It emphasized driving waste out of the manufacturing cycle. Toyota’s definition of waste includes items we don’t normally consider—such as inventoried parts, “work in process,” variable work load—as well as the usual defects, product returns and overbuilding [Muda, Muri, Mura].

Lean manufacturers look very different from historic American assembly lines, which emphasized repetition, motion analysis and economies of scale. Toyota instead promoted greater autonomation, the philosophy that repetitive activities should be done by machines, while thinking and creative activities should be done by humans. So Toyota’s factory floors included more robots, employing humans in teams to assemble complex systems, analyze problems and devise solutions. Since American assembly lines exploited speed and employee utilization, Americans rarely stopped the assembly line. In contrast, Toyota workers ceremoniously pulled an andon (“paper lantern”) cord to announce discovery of a defect: conveyor belts stopped, managers made their way to the location of the pulled cord, and then workers and managers rapidly performed root cause problem analysis, devised a short-term fix and initiated a plan to permanently avoid the problem.

Lean Manufacturing fueled Toyota’s rise and dominance from almost nothing. It produced only about 2000 passenger vehicles between 1933 and 1947. By 1984, Toyota was producing about 4 million cars per year, virtually all in Japan.

In 1984, under pressure to produce cars in the USA and fearful that it did not understand American workers, Toyota partnered with General Motors to use lean manufacturing in GM’s shuttered Fremont factory, which had a history of antagonism, sabotage and drug use. Toyota wanted to experiment with “the real thing,” dismissing GM’s suggestion they find other workers, and rehired the motley crew. After Fremont employees were trained in TPS, the factory showed dramatic improvements in quality and profitability; it produced the highest quality cars GM offered [This American Life, NUMMI (26 March 2010)]. Other GM plants failed to apply the approach (which still stupefies me), and GM declared bankruptcy in 2010. Ironically, GM closed the Fremont plant for good the same year. As if Fremont is the location of some strange singularity, Tesla, today’s most cutting edge mass production car manufacturer, bought the Fremont plant in 2010 and now produces its cars there.

Since 2012, Toyota has produced more cars and more profit than any other car maker worldwide.

I study lean manufacturing and its many contributions to our society. Some will note how many agile practices arise from lean manufacturing. In fact, agilists even see the same stupefying management behavior, such as reversing agile practices despite overwhelming data showing Agile’s superiority in their own organizations. Agile stole freely from lean, and virtually all agilists acknowledge lean’s power. Agile and Lean Manufacturing are pals who share a lot of toys.

But is Agile a Subset of Lean?

Recently a colleague asked me whether Agile was a subset of Lean. Could lean manufacturing people produce an agile software transformation? I’m a “big tent” agile guy. The Scrum Alliance designates me as a Certified Enterprise Coach, but I’ve often spoken at Lean/Kanban conferences. Lean/Kanban folks include many “quants,” like me: people interested in data and statistics. Anyone who wants to produce better products for more beneficiaries is in my tent. So if reality makes Lean the big daddy of Agile, I’m cool with that.

But Agile and Lean Manufacturing (not to mention software development and manufacturing) are different. Someone armed solely with a lean manufacturing background is extremely unlikely to produce a successful agile transformation, unless they embrace agile practices that are outside lean and understand how Agile promotes creativity.

Agile and lean manufacturing both take action “just-in-time,” but they support different things. One supports innovation, the other supports optimization. Unfortunately, sometimes those goals conflict.

The article that led Jeff Sutherland and Ken Schwaber to invent Scrum, the dominant Agile methodology, is Hirotaka Takeuchi and Ikujiro Nonaka’s, “The New New Product Development Game,” Harvard Business Review (Jan 1986). This article compares product design to the fast-moving, barely-controlled game of rugby. It sparked a revolution, so most agilists know of it.

churchill-rugby-ireland-2390383-600x350

“Scrum to England,” Boocal.

Scrum and Agile practices focus on product development (i.e., design) rather than manufacturing. Scrum supports creativity, learning and innovation rather than analysis, waste reduction and control. For example, the article discusses the six characteristics of designing products in this new way:

  1. Built-in instability
  2. Self-organizing project teams
  3. Overlapping development phases
  4. “Multilearning”
  5. Subtle control
  6. Organizational transfer of learning

Here’s how Wikipedia describes lean manufacturing:

Lean manufacturing or lean production, often simply “lean”, is a systematic method for the elimination of waste (“Muda”) within a manufacturing system. Lean also takes into account waste created through overburden (“Muri”) and waste created through unevenness in work loads (“Mura”). Working from the perspective of the client who consumes a product or service, “value” is any action or process that a customer would be willing to pay for.

Essentially, lean is centered on making obvious what adds value by reducing everything else. Lean manufacturing is a management philosophy derived mostly from the Toyota Production System (TPS) (hence the term Toyotism is also prevalent) and identified as “lean” only in the 1990s. TPS is renowned for its focus on reduction of the original Toyota seven wastes to improve overall customer value, but there are varying perspectives on how this is best achieved. The steady growth of Toyota, from a small company to the world’s largest automaker, has focused attention on how it has achieved this success.

Note that key line, “lean centers on making obvious what adds value by reducing everything else.” Lean applied in isolation doesn’t generate innovative designs fast, because its focus is to remove variation, randomness and the unexpected. Creative activity, where we design and build something never built before—exhibited more or less in virtually all software development—introduces variation inherently. Creativity and unpredictability often go together; the more innovative a creative act is, the more variability it produces. When careless lean advocates measure and control variation without leaving room for natural variation in creativity, they may throw out the baby with the bathwater.

Agile and Lean practices can work well together, when their contributions are respected—waste is created in software development (technical debt, manual testing, component team dependencies, etc.) and lean manufacturing strategies can help. In fact, virtually all Scrum courses include material on Kanban, Theory of Constraints and A3, which come from lean manufacturing. If lean manufacturing strategies dominate, however,  creativity and innovation, key aspects of software development, can suffer.

Very recently, “Lean Startup” appeared on the scene (2003 “Four Steps to the Epiphany” by Steve Blank, and 2011 “Lean Startup” by Eric Reis). Lean Startup accelerates innovation by discarding the idea that we have to create something with customer value, and that instead we seek market learning value first. Lean Startup is radical, and its application has created modern, constantly-experimenting behemoths such as Facebook. By learning more about the market, we waste less effort creating products people won’t likely buy or use. One could argue that “Lean Startup” is just a “very big lean manufacturing,” and from that newly expanded vision I might have to agree. However, lean manufacturing texts did not include the waste of “not meet customer demand or specifications” until [Womack 2003]. because they focus on designs that are already largely completed. 

Some agilists push Lean/Kanban, which is an adaptation of lean manufacturing for IT, as a software development practice. Lean/Kanban emphasizes none of Takeuchi and Nonaka’s six characteristics. Despite having appeared 40 years before Agile, lean manufacturing has gained little hold in software development. However, Lean/Kanban appears to work well for IT operations (configuration and deployment) activities. I attribute this to the lower need for innovation in operations.

When colleagues advocate Lean/Kanban in software development, and especially if they are transitioning from Scrum, I always ask them to institute a metric-driven rhythmic retrospective (which I discuss here), and ask them to compare their Kanban results to their Scrum over time. Without a rhythmic retrospective, I have found Lean/Kanban to be difficult to sustain, and I want organizations to help more people with higher quality products for a long time. I’ve seen some sad unravelings of agility when people selectively adopt Lean/Kanban practices; but when people include metric-driven Retrospectives, it seems to last and improve more rapidly.

With historic data supporting Agile, managers take significant risk when they contemplate turning back the clock, even for something as agile-friendly as lean manufacturing. Waterfall projects fail at a rate of about 29% and Agile projects fail at 9%; waterfall succeeds 14% of the time and agile succeeds 44% of the time; the rest are “challenged projects” that partially achieved their goals and are being used (Standish Group Chaos Report). 

This is the third time I’ve read the Takeuchi and Nonaka paper. Every time I do I find a new subtlety or insight. I’m surprised how prescient it was. It’s an easy read, and I’d invite you to read it yourself.

You might ask yourself, “Is Lean a subset of Agile?” I sometimes wonder that myself.

There is nuance here, I really do apply lean manufacturing techniques daily in my Agile work, and I welcome alternative views. If you comment, I’ll happily respond.

The post Is Agile a Subset of Lean Manufacturing? appeared first on LeadingAgile.

Categories: Blogs

Product Owner Survival Camp

Tyner Blain - Scott Sehlhorst - Wed, 03/30/2016 - 17:59

Product Owner Survival Camp 19-20 May, Cambridge, MA

Product owners are likely to find themselves alone in the organizational wilderness. Their organizations expect them to connect the towers of long-term strategic planning with the frontiers of great new products. Iterative and incremental development of solutions can bring these two worlds together. There’s always a gap between strategy and execution – and product owners are ideally positioned to help fill that gap.

What we need is a survival guide – a set of principles, tools, and techniques; learned and applied in a two-day “camp” with industry-leading experts in agile product management and product ownership.

Announcing Product Owner Survival Camp, USA – Cambridge MA 19-20 May 2016

Ellen Gottesdiener, Jeff Patton, and I (Scott Sehlhorst) have developed Product Owner Survival Camp USA to address exactly this challenge – helping product owners close the gap between strategy and execution.

Jeff Patton, Ellen Gottesdiener, Scott Sehlhorst - POSC Instructors

Building on the successful program developed and delivered in Europe in 2014 and 2015, we have designed this program to serve the needs and goals of product owners. We are bringing real-world expertise in agile product management and product ownership to you in this unique, action-packed program. You will learn ways to close the gaps between strategy and tactics. You will elevate your ability to discover the right product for the right customer at the right time, and tap into your product ownership leadership capabilities.

We’ve framed out major elements in closing the gap from strategy to execution, and we’re excited to be sharing this in a two-day program in Cambridge, MA, on May 19th / 20th.

In addition to clarifying  the role of product owner and focusing on the challenges you face as a product owner, you will benefit by learning to:

  1. Conquer the chaos of backlog bloat and story hell
  2. Identify and vet new product opportunities in a way that invites, excites, and involves customers and internal stakeholders
  3. Convert strategy to reality by prioritizing the right strategic goals to define an agile product roadmap and manage a lean product backlog
  4. Convince internal stakeholders—from execs to engineers—why your investment choices are valuable and feasible
  5. Overcome requirements confusion and conflict by holistically exploring product options and making transparent decisions
  6. Lead teams and stakeholders toward shared product outcomes through dynamic collaboration

The Product Owner Survival Camp USA (POSC-USA) program provides you with essential skills to deliver high-value products. You will explore and practice powerful ways to align and collaborate to co-create valuable products that embrace the disciplines and philosophy of Agile/Lean product discovery and delivery. Evolve from surviving to thriving!

Join us in Cambridge, Massachusetts on May 19-20 for this unique two-day learning experience that combines the best features of a conference and a training class.

Registration is now open. For more information, visit http://productownersurvivalcampusa.com/

#productowners can help close the gap between strategy and execution bit.ly/poscusa
Categories: Blogs

Decluttering Photographs: A Scrum Mindset

Learn more about transforming people, process and culture with the Real Agility Program

A common Scrum myth is that Scrum teams don’t keep documents. The many sticky notes on the wall of a scrum room can be anxiety-provoking to traditional suit-and-tie folks. Formal documentation is considered to be the hard product of real work. But the truth is two-fold.

First, Scrum teams do keep documents: The product backlog and the sprint backlog, which are two of Scrum’s artifacts, are key documents for a Scrum project. Each has a precise role in moving the scrum team from product conception to shippable increments. Different teams or organizations keep these artifacts in formats that suit them, but the point is that they do.

Second, it is not true that all formal documents are a faithful (or useful) reflection of work. More often than not, it is not clear what they’re for. Team members typically keep documents in the hundreds, outdated files are seldom deleted and clean versions are often mixed up with drafts. As with all clutter, the 80-20 rule applies rather well here: 20% of documents supports 80% of the work effort. So, 80% of formal documentation is in fact clutter.

Adopting Scrum will challenge organizations to focus on nothing other than completing deliverables of highest value to customers. This focus is in fact reflected by the few but super-useful documents that Scrum teams keep. But to move into this state of razor-sharp focus, we need a clutter-free environment. And while there are many forms of clutter on the job, documents are a big one. Why? Because the documents we keep and the way we use them reflect how well we understand our role. Much like a divinatory tool, the collection of documents you keep are very telling of how you do your work. Is there an orderly process as would be reflected by an orderly and well-organized set of a few focused documents? Or is it a firefighting role as would be reflected by a random set of documents contrived into folders?

Decluttering work documents compares rather precisely with decluttering photographs. Both clutter categories bring up very similar levels of emotional and intellectual charge as well as frustration with the sheer volume that needs to be processed and purged. And unlike the ease of throwing out something bulky like a broken chair or a dulled out piece of clothing, decluttering documents and photographs are very daunting tasks whose detail will bring anyone to tears.

Office documents are very much like a typical photo collection by enthusiastic parents: Several hundred photos of Maxi kicking ball, three dozen photos of the tenth birthday cake, one whole album of Auntie over for dinner, and so on. Photos will range from the incognito-fuzzy to the acceptably crisp. All are equal, however, and are kept in bulging pounds of photo albums or hard drives ranging in the terabytes – or both.

You can see the similarity with documents: Several hundred versions of the scorecard, three dozen presentations on the same topic, a few thousand archive files, and so on. ‘Fuzzy’ documents surely make up the majority. They include the many versions with corrections, inputs and errors, and the older versions to name just a few. The ‘crisper’ documents are typically the most recent ones and likely the ones shared with the VP. But these soon become ‘fuzzy’ as new versions are produced. The main issue with documents is knowing exactly what they’re for and how they support value-adding processes; a problem that doesn’t exist in a Scrum framework.

The main thread of comparison between photos and work documents is the degree of psychic severance that must take place in the mind in order to be capable of throwing out the useless lot. In truth, decluttering photographs is one of the hardest categories to tackle and does not happen until one has built a strong decluttering muscle with simpler categories like clothes, shoes, pots, pans and the like. Because photographs fall under the ‘sentimental’ umbrella, it takes much more than simply assessing functionality. Instead, it requires:

  1. An assimilated understanding of how to declutter, which means having built the decluttering muscle with simpler categories of stuff and having the gut to make sharp decluttering decisions,
  2. A willingness to look squarely at the past and contrast it with today, which basically means grieving and all the complex steps that alone involves, and
  3. A decision about what will be taken into the future, which must be actively considered as opposed to simply assuming that it’s what remains after decluttering the unwanted.

Releasing the way we create, share and keep documents is tantamount to releasing the way we work. That’s why decluttering documents is loaded with anxiety about how the future modus operandi will look like. An additional consideration here is to do this before anything is archived because if all the old is just moved into an archive, no one will open it because everyone subconsciously knows of all the mold that’s growing in there. So better not postpone the pain and roll with the punches. Taking a sword at opening each and every single file and processing for keep-dump-or-change, takes a lot of gut and staying power. Our attachment to documents is quite entrenched but it must be severed for the sake of agility.

The result of genuinely decluttering photographs is astonishing. The grieving process can be very intense; not only are issues resolved and dropped but pieces of the past are recalled into the present and the spirit regains integrity. The past is forgiven and the remains of the day are a powerful collection of sharp moments that are very much alive. Photographs are no longer nostalgic snapshots of the past but evidence of an assimilated experience in effect today. And with that, something very special happens: Fewer and fewer photographs are taken as soaking up a moment becomes far more precious than clunking through photographing it. This vulnerability to the passing of time is a key ingredient for living life intelligently.

Can you draw the parallels with documents? We spend most of our days at work. So, let me invite you to start tackling your collection and see how your newfound awareness pours into every other area of your life.

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!
facebooktwittergoogle_plusredditpinterestlinkedinmail

The post Decluttering Photographs: A Scrum Mindset appeared first on Agile Advice.

Categories: Blogs

Agile Retrospectives: Hoe vaak doe je ze?

Ben Linders - Wed, 03/30/2016 - 14:24
Agile teams gebruiken retrospectives om hun manier van werken te reflecteren en te verbeteren. Veel Scrum teams sluiten iedere iteratie af met een retrospective. Teams die Kanban gebruiken doen veelal wekelijks een retrospective of doen zo vaak als nodig is een mini-retrospective om van een probleem te leren en in kleine stapjes te verbeteren. Hoe vaak doen jouw teams agile retrospectives? Continue reading →
Categories: Blogs

Agile and Lean Program Management is Done

Johanna Rothman - Wed, 03/30/2016 - 13:50

I sent my newsletter, Scaling Agile and Lean to Programs to my subscribers yesterday. (Are you one of them? No? You should be!)

If you are trying to use agile for several projects that together deliver value (a program), you might be wondering what the “right” approach is. You’ve heard of frameworks. Some of them seem to be a bit heavy.

Instead of a framework, consider your context. You and your organization are unique. Do you have hardware to integrate into your product? Do you have agile and non-agile teams who are supposed to deliver? Are you trying to work in iterations and they don’t quite work at the problem-solving level?

 Scaling Collaboration Across the OrganizationYou have many choices. In Agile and Lean Program Management: Scaling Collaboration Across the Organization, I offer you options for how to think about and solve these and many other problems. The book is principle-based, not practice-based. That way, if you consider the principles, you’ll be in great shape, regardless of what you decide to do.

Please do check out the book. It’s available everywhere fine books are sold. (I love saying that even if it is passive voice!)

Categories: Blogs

Managing Dotfiles With Ansible

Yesterday I posted about managing our local configuration with Ansible and today I’m going to continue this path by putting my zsh configuration under configuration management.

Installing ZSH and oh-my-zsh

First up let’s install our preferred shell and customizations. For me this is zsh and oh-my-zsh. Up front I know that this is going to probably be a multi-step process so I’m going to create a local role to bundle up the tasks and templates we’ll be using. I create a new directory named roles and add it to the roles_path in our ansible.cfg. This new directory with our new role will look like the following.

For our initial set of tasks we’ll install zsh via homebrew, configure zsh as the current user’s shell and install oh-my-zsh.

You can see this change in its entirety here. Next up we’ll add a template for our zshrc that we can customize to our liking. To start we’ll grab the zshrc template from the oh-my-zsh git repository and save it to jamescarr.dotfiles/templates/zshrc.j2.

A good first piece of literal text to template out is the zsh theme and the plugins loaded up. We’ll define these with default variables under jamescarr.dotfiles/defaults/main.yaml.

You’ll notice here we’ll also be polite by backing up the existing zshrc file if it is present. A good benefit with this example is that we can now switch up the different pieces of our .zshrc configuration from our playbook by overriding the default variables.

You can find everything up to this point at this referenced commit in jamescarr/ansible-mac-demo.

Up Next

Obviously a topic like zsh customizations is a rather large undertaking deserving of its own post so tomorrow I’ll share some of my personal zsh functions and aliases that I find extremely useful to start off with.

Well did you find the useful? Did you run into any problems following along? Please let me know in the comments below!

Categories: Blogs

Why Messaging? Why Not Have The Web Server Handle The Request?

Derick Bailey - new ThoughtStream - Wed, 03/30/2016 - 13:30

A reader of my RabbitMQ Patterns email course recently asked a question about using messaging systems within an web application. Specifically, they wanted to know why you would use a request/reply pattern over a messaging server, instead of just handling the request within the HTTP call and the web server, itself.

The short answer is, “it depends”.

Http or rabbitmq

There are times when the request can and should be handled by the web server, directly. Loading a blog post to view? Yeah, that’s probably a good call for the web server to handle. But there are also times when it doesn’t make sense for the web server to handle the workload, directly.

The following is a short conversation that took place via email, regarding messaging vs HTTP.

Shifting The Workload

Them:

What are the benefits of using a request / response queue over just doing the work as part of http request?

For fire and forget messages I can see the performance benefits…

Is it just being able to shift the work load to another service? Or abstract some complexity of the work?

Me:

That’s the core of it, really – shifting the workload to another service.

If you have an external system that contains the information you need, or can do the work within a few hundred milliseconds, it might make sense to do request/reply.

Scheduled Code Example

Me, continued:

For example, I have a system that uses a node.js module for running tasks at specific times. It’s a bit like a cron job, but it runs entirely in node.

This module lives in a back-end service that runs in it’s own process. I used a separate process because this service has nothing to do with handling HTTP requests and should not live in the HTTP context or web server, directly.

But, my web app needs to get a list of what’s running next, and display them on a particular page. So how do I do that? What are the options?

Read The Database Directly

I could read the database collection that the scheduler module uses – it’s in the same database that the web app uses. But, that would be a backdoor into the functionality the module provides. I would have to re-create the logic of the scheduler module within my web app, to translate the data into something meaningful for the user.

No, thanks.

A New Schedule Module Instance

I could create new instance of the scheduler module in my web app. But this would create two sources of the truth on what is running next, and they would be in conflict with each other.

Each instance would assume it owns the data in the database (because the instances don’t know about each other). This would cause problems when one updates the database and the other tries to update it again.

Again: No, thanks.

Request/Reply

The solution, in this case, was to use request/reply to get the list of items from the existing back-end service. That way I have a single source of the truth for the scheduled items, and don’t have to re-create logic from an existing module.

Decoupling The Database

Them:

Ah thanks, that helps. Nice, interesting idea. Means you web app isn’t so coupled to the database.

Me: 

Exactly!

There’s a saying I heard a long time ago, “a database is not an integration layer.”

I don’t remember where I first heard that, but it continues to ring true after many years.

I could have tried to use the database as an integration layer, but that would have added complexity and potential problems to the system. 

It made more sense for me to use messaging in this scenario, and allow the one source of truth on the upcoming schedules to be that one source of truth.

 

Beyond Reading: Writes With Messaging

There are far more examples of what can (and should) be done with messaging, when handling HTTP requests. The list is nearly endless, and can go so far as to say database writes should be handled through messages!

In my RabbitMQ For Developers training package, I spoke with Anders Ljusberg about Event Sourcing and messaging systems for database writes.

It was an enlightening conversation, to hear about the need for pushing a database write to a back-end service, and how this affects the over-all architecture of a web application.

Be sure to check out the complete RabbitMQ For Developers course, for this and other interviews, screencasts, ebooks and more!

Categories: Blogs

Scaling Scrum with The Nexus Framework

TV Agile - Wed, 03/30/2016 - 09:57
In this video SSW Chief Architect Adam Cogan discusses with Scrum.org Team Member Steve Porter about the Nexus Framework, an Agile framework designed for large scale software development. The Nexus framework is presented by Scrum.org as a framework that drives to the heart of scaling: cross-team dependencies and integration issues. It is an exoskeleton that […]
Categories: Blogs

Links for 2016-03-29 [del.icio.us]

Zachariah Young - Wed, 03/30/2016 - 09:00
Categories: Blogs

Agile Change or Adoption Always Starts with Why

Notes from a Tool User - Mark Levison - Wed, 03/30/2016 - 07:08

Your organization has decided to become more “Agile.” Why? As we learned in a previous blog post, “Because Our Competitors Are” is not a valid – or sensible – reason.

Before embarking on a change, adoption, or improvement program, you need to know the rationale behind that decision. So… why Agile?

Group of peopleA traditional approach to answering this question might see the executive team going off-site for two to three days and holding a workshop where they decide why they should be Agile, then design an adoption strategy, and then summarize the whole thing in a few sentences to be sent out in a memo.

Typically, large-scale change initiatives have a lot more ceremony, more meetings, and more setup than this. However, there are several key failings, including that they involve only a select few executives in the envisioning and decision-making process, and they attempt to plan for the long haul.

There are dozens of examples in our industry of failed change efforts that have cost billions of dollars and proved that this approach doesn’t work. At Nokia, Stephen Elop issued the famous ‘burning platform’ memo in 2011, and yet two years later the company was sold to Microsoft. In 2013 Avon had to write off $125 million[1] of work that built an enterprise software implementation which drove representatives away. This was change that failed to help the very people it was intended for.

These and other failures involve some combination of the following:

  • Why – The “Why” isn’t understood by most of the victims of change.
  • Strategy – The “Strategy” created by the executive group doesn’t make sense to all of the people doing the work.
  • Ownership – People at the edges of the system (who do most of the work) feel no ownership of the change.
  • Connection – The strategy doesn’t appear connected to the problems that the people at the edges of the system are experiencing.
  • Improvement -The strategy appears to improve the lot of the executives, but not of the doers.
  • Culture – The change doesn’t fit the organization culture.
  • Leadership – Top level is asking for change but doesn’t appear to be involved in making it happen.

To be effective, Agile organizational change needs to… well, involve the Organization! Not just the executives who have made the decree, often without fully understanding what the goals of the change are. This shouldn’t be a quick decision made at a two-day corporate retreat. It needs to be an ongoing effort to figure out the “why” collaboratively and share it effectively, being mindful of some essential ingredients.

We address those ingredients in the next blog post: Agile Change or Adoption – the Steps to Go from “Why” to “How”

[1] Avon’s Failed SAP Implementation A Perfect Example Of The Enterprise IT Revolution – Ben Kepes: http://www.forbes.com/sites/benkepes/2013/12/17/avons-failed-sap-implementation-a-perfect-example-of-enterprise-it-revolution

Image attribution: http://photodune.net/

Categories: Blogs

Managing Your Macbook with Ansible

For a long time I’ve been a big believer in Infrastructure as Code and I have always wanted to use configuration management to provision my personal workstation and keep it constantly updated to an expected state. Boxen was one of the first tools I saw in this space and it even seemed like it might be comfortable since I was using Puppet at the time. However I never really had a lot of luck with it and the original aim of Boxen was actually lost on us at Zapier since we engineered a very nice docker-compose based setup that lets anyone begin running and hacking on zapier locally constrained by the time it takes to download the docker images for the first time.

That being said when we began transitioning from Puppet to Ansible last year and I naturally started using it locally to kind of whet my appetite a bit. Here’s a brief run down of how I’m currently using Ansible to manage my laptop and some notes on where to go next.

Getting Started

There are several guides out there on getting Ansible installed, the most authoritative being the instructions right on Ansible’s website. I won’t repeat those well written steps here.

Once that’s all done let’s run ansible --version and verify we’re running Ansible 2.0.1.0 or above. If you’re visiting from the future then I will have to say that I am really unsure if this blog post will work with 3.0.0 or whatever release is out then. Keep in mind this post is written in 2016. 🙂

First up we’ll create a very simple Ansible playbook that just prints a hello world message to the console to ensure we have everything configured correctly.

Place this in a project directory (I name mine “personal-dev-laptop”) and run ansible-playbook main.yml. If all is configured correctly you’ll see a playbook run that executes a task that prints out “Hello World” to the console.

Homebrew

The most important piece to a provisioning system is the package management and Ansible is no different. Homebrew is the go to on OSX and thankfully Ansible has a pretty decent homebrew module for managing the state of different Homebrew packages. Let’s dip our toes in by adding a task to ensure macvim is installed and at the latest version.

The nice benefit here is that each time we run Ansible macvim will automatically get updated to the latest available package. However if we want to ensure a package is simply installed but don’t want to upgrade each time we run we can set the state to `present`. After awhile if we’ve worked with vim and decided that it’s just not for us and we’d prefer to use emacs instead we could just set macvim’s state to absent and emacs state to latest.

Taking It Further
Sure we can just keep adding our own tasks to install roles, perhaps even using a with_items iterator to include a big list of them but sooner or later we’re going to be duplicating a lot of work someone else has done. Which is a good time to introduce third party roles installed via ansible galaxy. There are most likely several good roles out there but my favorite so far is geerlinguy.homebrew. I usually put a requirements yaml file in the main root of my project with the module I want to use and the version I want to lock in.

Now to install this third party role we’ll run ansible-galaxy install -p vendor -r requirements.yaml. The -p switch will install it to a local directory named vendor so we don’t clobber the global include path and we can add that directory to our project’s .gitignore so that it isn’t stored in git. We also add an ansible.cfg to specify the role path for third party roles we’ll be using.

Now we also update our main.yaml to include a few changes. Firstly we want to include the new role we just imported and then we move the packages we want to install as variables that the homebrew role will utilize.

This time we’ll run with the -K switch since this role also ensures that homebrew is installed and will require sudo access to do so. Now I know what you’re thinking… you’re thinking “James is trying to hack my box!” and quickly closing your browser tab. Obviously you should never provide sudo without giving the source code a look over and the most important pieces will be the main task file and meta file where there could be dependent roles. After careful inspection we decide all is good and run ansible-playbook -K main.yml. Congratulations, you now have Spotify and iterm2 installed!

One small improvement to make before we move on is to extract these variables that are specifically for homebrew to their own var file. While it might seem silly now, sooner or later we might be using many roles that utilize many different variables and mixing them will lead to a lot of confusion. I personally like to name the variable files after the role they’re used for as illustrated below.

Managing OSX Settings

You can do a lot of tweaking to how your OSX behaves by using the osx_defaults module to manage OSX Defaults. There’s a lot of opportunities here but I’ll just leave a quick and dirty example to set our preferred screensaver below.

You could possibly even go as far as using this to manage various applications you have installed and possibly even setting registration keys for those applications. I haven’t even gotten to that point yet either so I’m not covering it here.

Further Reading

Well I hope this was good for you… it was good for me and helped me flesh out some of my current setup. I’m still on my path to learning how to best utilize ansible to manage my development environment so there’s definitely more to learn that I’ll continue to share as time progresses. I’m also not ignorant of a few other projects that aim to make working with ansible to manage development environments easier and one I’ve been looking at is Battleschool.

You can find the completed work for this blog post on github at jamescarr/ansible-mac-demo.

Categories: Blogs