Skip to content

Feed aggregator

Agile Cymru, Cardiff Bay, UK, July 5-6 2016

Scrum Expert - Tue, 06/21/2016 - 10:00
Agile Cymru is a two-days Agile conference organized by Agile Wales. The 2016 edition will take place July 5 and 6. Agile Cymru offers practical advice, techniques and lessons from practitioners, experts and beginners in the field of Agile software development and project management with Scrum. In the agenda of the Agile Cymru conference you can find topics like “How Deep are your Tests”, “How to Grow Beautiful Teams”, “The Power of Feedback Loop”, “Agile Product Roadmap”, “Governance and Agile”, “Scrum is Good but Kanban is Better”, “How to fail a software project fast and efficiently?”, “Game of Scrums: Tribal behaviors with Agile at Scale”, “Dreaming – how business intent drives your Agile initiatives”, “From Agile projects to an Agile organization – The Journey”, “Whatever Happened to Being Extreme?”. Web site: http://www.agilecymru.uk/ Location for the AgileCymru conference: Wales Millennium Centre, Bute Place, Cardiff Bay CF10 5AL, UK
Categories: Communities

Agile and Scrum beyond Software Development

Ben Linders - Tue, 06/21/2016 - 09:43

Agile and Scrum Beyond Software Development - Ben LindersIn one of my workshops I explored how to use agile and Scrum beyond software development, for example in marketing, sales, support and in management. I gave this workshop in-house for a client to an audience which was not directly involved in software development. Many good ideas came up when preparing and giving this workshop, in this post I’m sharing them with the agile community.

Over the years I’m using agile principles and practices from Scrum beyond software development. For example, I do process improvement with agile, use Scrum for process improvementreduce process debt with agile, manage projects using agile, and use agile to prepare and give training and workshops. I wrote about the golden rules for agile process improvement. I was also involved in eduScrum, an initiative where Scrum is used by students for education at school.


advertisement:

Retrospectives Exercises Toolbox - Design your own valuable Retrospectives

After giving a keynote at the 1st Conference in Melbourne I watched a talk from Eduardo Nofuentes about adopting agile beyond software. He slightly modified the agile manifesto to broaden it’s purpose while keeping the original thoughts:

We are uncovering better ways of working by doing it and helping others do it. Through this we have come to value:

Individual and interactions over processes and tools
Outcomes over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

I’m convinced that agile can be applied in many different areas. But, the information supporting this is scattered and sometimes hard to find, and that makes it difficult to use it and apply agile in other areas. Then the opportunity arose to bundle stuff … Help from the Scrum Master Community

A client asked me to give a workshop about agile to an audience which was outside software development. Explaining the agile manifesto and concepts from Scrum in my opinion wouldn’t be enough. I wanted to show actual examples of using agile beyond software, both from my own practice and cases published by others.

I used my network and posted a question on Facebook in the Scrum Master Community:

I’m preparing a workshop on agile/Scrum for non-IT. Anything that I should include, like experience reports, reference websites, checklists?

I got a lot of good ideas and experiences, which are in my workshop, next to what I do with agile and Scrum. In return for the contributions I promised to publish an overview of the workshop; to share an extract of my sheets which includes examples from using agile and Scrum beyond software development. Well, here it is:

Agile and Scrum beyond Software Development (overview) – Ben Linders from Ben Linders

A big thanks to everybody who provided ideas, on Facebook or directly, or by publishing examples on the internet on how they used agile and Scrum. All of this is very inspiring, and helped me to compile the sheets for the workshop which helped my client to take the first steps towards increasing their agility. Tailoring to use agile and Scrum beyond software

Of course you have to tailor agile and Scrum to support your needs. This is true when you use them in software development, but even more if using them beyond. Things to consider are:

  • Deliveries & Goal: What is it that you want to reach? Which agile principles and Scrum practices can help you to get there? How?
  • Custumers & stakeholders: Who do you have to involve to reach your goals and be able to deliver?
  • Team & collaboration: How do you want to work together? What needs to be arranged to make it possible?
  • Define “Done”: Which criteria need to be satisfied before something is good enough to deliver? Which existing processes and practices can be used?
  • How to get feedback: What can you do to know if you’re moving in the right direction (or not)? Which feedback do you need, from whom?

Summing up: Yes, you can certainly use agile and Scrum beyond Software Development. The agile mindset and principles are general enough to use them in many areas. Share your experiences!

There are many good examples available. I’ve listed the ones I know, but there are probably many more. Please share your experiences by commenting to this blog post if you have used agile and Scrum outside software development. Let’s collaborate to learn from each other, and help people around the world to improve their way of working using agile and Scrum!

Categories: Blogs

How to Keep flowtype Running and Report Errors on Save

Xebia Blog - Tue, 06/21/2016 - 09:00
We use flow from Facebook to run type checking on our codebase. When you run ‘flow status’ it starts a flow server in the background and keeps it running. That way after the first run the results of each next run are almost instant. The only thing currently lacking is a watch mode, but there
Categories: Companies

Links for 2016-06-20 [del.icio.us]

Zachariah Young - Tue, 06/21/2016 - 09:00
Categories: Blogs

Performance, Not Policy

Evolving Excellence - Tue, 06/21/2016 - 00:01

No-PoliciesFew people realize how employee policy manuals, usually given to you on your first day and then mostly forgotten, shape an organization’s culture and thereby its fundamental performance.

To give you a reference point, one company I worked for had a forty-plus-page employee manual that started every section with “COMPLIANCE IS ESSENTIAL” highlighted in bold, with “required to conform” sprinkled liberally throughout the document. The manual ended with a meaty discussion of the punitive measures that would happen if someone deviated from the policies. And this was a company considered very innovative in many ways!

The other extreme is Zaarly, a San Francisco-based startup. Its employee handbook, posted online for even non-employees to see, talks directly about culture. The “Rules for Work” section begins with “We don’t have these.” And in a style prevalent throughout the document, it adds that “if you want to coast, we recommend you apply for a job at Craigslist.” Included are some good thoughts on teams, work, and communication, but no rules.

Another example is the famous Netflix “business culture” PowerPoint that serves as the company’s employee handbook. Similar to Zaarly’s handbook, it talks a lot about culture and a lack of rules. There is no vacation policy, and the travel and expense policy is literally five words: “Act in Netflix’s best interests.” That’s it. Unlike Zaarly, Netflix does say some rules are necessary, such as: “Absolutely no harassment of any kind.” In this case, I completely agree, especially on that item. Some topics relating to privacy, security, and regulatory requirements are important enough that they need to be spelled out in no uncertain terms.

Netflix believes high-performance people should be free to make decisions, and those decisions need to be grounded in context. Mission, vision, and value statements do not create context. To demonstrate this, Netflix’s presentation provides the example of how Enron’s value statement included “integrity.” Real company values are shown by who gets rewarded for embodying desired behaviors and skills. The document goes on to describe the primary Netflix values and the associated behaviors.

At Netflix, flexibility is more important over the long term than efficiency. To inhibit the chaos that too much flexibility in a large organization can create, the company hires (and keeps) only high-performance people. High-performance people make great decisions, so building a staff of them is better than having people who are good at following lists of rules. Later on in the Netflix document, there is a good discussion that encourages managing with context instead of trying to control people. That way, when something fails, managers look to figure out what went wrong with the process rather than with the people.

One part of the Netflix document that gave me pause was an insinuation that defined processes (such as standard work) are all bad. But doing standard work doesn’t necessarily mean the employee has zero flexibility. As those of us in the Lean world know, standard work is the foundation for kaizen. Once an employee deeply understands a process, he or she can (and is expected to) come up with ways to improve it and then share it, which is called yokoten.

Gemba Academy has adopted many similar concepts in our own Gemba Academy Culture Code.

In January 2014, Brad Power posted a piece in Harvard Business Review titled Drive Performance by Focusing on Routine Decisions that hits at a similar concept. Instead of creating rule-bound defined processes, companies should focus on improving the quality of the decisions made by managers. Power illustrates the idea with an example those of us in the manufacturing world have all experienced: the maelstrom of materials control. He describes how the materials department of an electronics distributor was able to improve operations by better training managers to make key decisions about inventory. The goal of the training was to get managers to focus less on perfecting company processes (the “box and arrows” of a flowchart) and more on understanding what objectives the processes were supporting in the first place. When managers were able to understand how the processes affected actual business performance, they were able to make decisions (the “diamonds and arrows”) that improved performance:

These two stories highlight the advantages of focusing process improvement on “diamonds and arrows” — i.e., making better decisions. Project leaders who focus exclusively on the “boxes and arrows”of workflow action improvement will often find themselves caught up fixing yesterday’s operations and systems issues. Workers who participate in these interviews and workshops tend to fixate on the pain points they want fixed. This focus on immediate problems can actually distract the project team from the real goals of the business and the decisions that will help achieve them.

Are your rules improving the boxes (company processes) but harming the diamonds (managers’ decision-making)? How is that rigidity affecting your long-term performance? Do you have a team of high-performance people that you can trust to deal with the diamonds in a flexible, agile way? And how do your under-lying documents, even down to the employee handbook, support or impede that? These are questions to consider as you look to improve your company’s performance.

Categories: Blogs

10 Lessons from a Long Running DDD Project – Part 2

Jimmy Bogard - Mon, 06/20/2016 - 21:04

In Part 1 of this 2-part series, I walked through some lessons learned from the first incarnation of our project. The original project I’d still qualify as a success, in that it was delivered on-time, within budget, and is still under active development today. But we learned a lot of lessons from that project, and were lucky enough to have another crack at it so to speak when we started a new project, in the almost exact domain, but this time the constraints were quite a bit different.

In the first project, we targeted everyone that could possibly be involved with the overall process. This wound up to be a dozen state agencies and countless other groups and sub-groups. Quite a lot of contention in the model (also a great reason why you can never have a single master data model for an entire enterprise). We felt good about the software itself – it was modular and easy to extend, but the domain model itself just couldn’t satisfy all the users involved, only really a subset.

The second project targeted only a single aspect of the original overall legal process – the prosecution agency. Targeting just a single group, actually a single agency, brought tremendous benefits for us.

Lesson 6: Cohesiveness brings greater clarity and deeper insight

Our initial conversations in the second project were somewhat colored by our first project. We started with an assumption that the core focus, the core domain would be at least the same as the monolith, but maybe a different view of it. We were wrong.

In the new version of the app, the entire focus of the system revolves around “cases”. I know, crazy that an app built for the day-to-day functions of a prosecution agency focuses centrally on a case:

image

Once we settled on the core domain, the possibilities then greatly opened up for modeling around that concept. Because the first app only tangentially dealt with cases (there wasn’t even a “Case” in the original model), it was more or less an impedance mismatch for its users in the prosecution agency. It was a bit humbling to hear the feedback from the prosecutors about the first project.

But in the second project, because our core domain was focused, we could spend much more time modeling workflows and behaviors that fit what the prosecution agency actually needed.

Lesson 7: Be flexible where you need to, rigid in others

Although we were able to come to a consensus amongst prosecution agencies about what a case was, what the key things you could DO with a case were and the like, we couldn’t get any consensus about how a case should be managed.

This makes a lot of sense – the state has legal reporting requirements and the courts have a ton of procedural rules, but internal to an agency, they’re free to manage the work any way they wanted to.

In the first system, roles were baked in to the system, causing a lot of confusion for counties where one person wore many different hats. In the new system, permissions were hard-coded against tasks, but not roles:

image

The Permission here is an enum, and we tied permissions to tasks like “Approve Case” and “Add Evidence” and “Submit Disposition” etc. Those were directly tied to actions in our application, and you couldn’t add new permissions without modifying the code.

Roles (or groups, whatever) were not hardcoded, and left completely up to each agency how they liked to organize their work and decide who can do what.

With DDD it’s important to model both the rigid and flexible, they’re equally important in the overall model you build.

Lesson 8: Sometimes you need to invent a model

While we were able to model quite well the actions one can perform with an individual case, it was immediately apparent when visiting different county agencies that their workflows varied significantly inside their departments.

This meant we couldn’t do things like implement a workflow internal to a case itself – everyone’s workflow was different. The only thing we could really embed were procedural/legal rules in our behaviors, but everything else was up for grabs. But we still wanted to manage workflows for everyone.

In this case, we needed to build consensus for a model that didn’t really exist in each county in isolation. If we focused on a single county, we could have baked the rules about how a case is managed into their individual system. But since we were building a system across counties, we needed to build a model that satisfied all agencies:

image

In this model, we explicitly built a configurable workflow, with states and transitions and security roles around who could perform those transitions. While no individual county had this model, it was the meta-model we found while looking across all counties.

Lesson 9: Don’t blindly follow pattern advice

In the new app, I performed an experiment. I would only add tools, patterns, and libraries when the need presented itself but no sooner. This meant I didn’t add a repository, unit of work, services, really anything until an actual pain surfaced. Most of the DDD books these days have prescriptive guidance about what your domain model should look like, how you should do repositories and so on, but I wanted to see if I could simply arrive at these patterns by code smells and refactoring.

The funny thing is, I never did. We left out those patterns, and we never found a need to put them back in. Instead, we drove our usage around CQRS and the mediator pattern (something I’ve used for years but finally extracted our internal usage into MediatR. Instead, our controllers were pretty uniform in their appearance:

image

And the handlers themselves (as I’ve blogged about many times) were tightly focused on a single action, with no need to abstract anything:

image

I’ve extended this to other areas of development too, like front-end development. It’s actually kinda crazy how far you can get without jQuery these days, if you just use lodash and the DOM.

Lesson 10: Microservices and anti-corruption layers are your friend

There is a downside to going to bounded contexts and away from the “majestic monolith”, and that’s integration. Now that we have an application solely dealing with one agency, we have to communicate between different applications.

This turned out to be a bit easier than we thought, however. This domain existed well before computers, so the interfaces between the prosecution and external parties/agencies/systems was very well established.

This was also the section of the book skipped the most, around anti-corruption layers and bounded contexts. We had to crack open that section of the book, dust it off, smell the smell of pages never before read, and figure out how we should tackle integration.

We’ve quite a bit of experience in this area it turns out, so it was really just a matter of deciding for each 3rd party what kind of integration would work best.

image

For some 3rd parties, we could create an entirely separate app with no integration. Some needed a special app that performed the translation and anti-corruption layer, and some needed an entirely separately deployed app that communicated to our system via hypermedia-rich REST APIs.

Regardless, we never felt we had to build a single solution for all involved. We instead picked the right integration for the job, with an eye of not reinventing things as we went.

Conclusion

In both cases, I’d say both our systems were successful, since they shipped and are both being used and extended to this day. With the more tightly focused domain in the second system we were able to achieve that “greater insight” that the DDD book talks about.

In case anyone wonders, I intentionally did not talk about actors or event sourcing in this series – both things we’ve done and shipped, but found the applicability to be limited to inside a bounded context (or even more typically, a corner of a bounded context). Another post for another day!

Categories: Blogs

Integrating Targetprocess with other tools via Tasktop Sync

TargetProcess - Edge of Chaos Blog - Mon, 06/20/2016 - 16:33

If integration with another software tool is absolutely essential to your work, you’ll be happy to hear about our latest partnership. You can now synchronize your Targetprocess account with any tool on Tasktop Sync, including Atlassian JIRA, Microsoft Team Foundation Server (TFS), GitHub Issues, Zendesk, HPE Quality Center, Bugzilla and more. You can find a full list of possible integrations here.

This means that if you or any of your teams are using a separate tool or software infrastructure for their work – perhaps for quality assurance, product support, ALM, or PPM – you can use Tasktop Sync to automatically synchronize data between the multiple solutions. Teams can work with the tool of their choice, without having to waste time manually transferring and standardizing information.

How will this help Targetprocess users?

Let's start with an example: You and your company have just made the switch to Targetprocess, but QA has requested another year with their current tool. You can use Tasktop Sync to link entities in Targetprocess with whatever artifact is used to represent bugs in your QA team's tool.

This kind of synchronization allows Targetprocess to show the real-time status and priority of work items, and can also help your teams maintain flow and traceability. Keep in mind that this is just a singular example; Tasktop Sync can do far more than just synchronizing cards. You can link an entire software infrastructure to Targetprocess, if need be.

  • Seamlessly & bi-directionally integrate with other PPM, Agile planning, requirements management, test management, and service desk tools
  • Save time by eliminating the need for import-export of data and manual standardization of information between tools
  • Synchronize artifacts across the lifecycle
  • Reduce time to market with better flow and traceability between tools
  • Get real-time data for faster and more informed decisions
  • Keep all teams on the same page, even if they work from different tools
How to setup integration with other tools via Tasktop Sync

To get started with Tasktop Sync, go to the Tasktop website and fill out this form. For an annual fee (the amount of which depends on your requirements), they will sync your Targetprocess account with the  tools or infrastructure you require (if the software is included on Tasktop Sync’s integration list). You can view Tasktop’s installation guide here.

More information on Tasktop

Tasktop is a market leader in the Software Lifecycle Integration market. Unlike other common integration providers, there is no charge each time an integration is used, making this solution ideal for users who require in-depth infrastructure synchronization or are frequently switching between tools. The service helps to improve end-to-end traceability for companies that use multiple tools for collaboration or infrastructure.

Notable Tasktop Features: 

  • System administrators can integrate software tools through a graphical interface that lets them create, maintain and monitor integrations – with no programming.
  • Differences among data types, text formatting, and even very complex relationships among artifacts are handled right out of the box.
  • ALM tool users never see Tasktop Sync and don’t leave their tool of choice; artifacts are synced automatically without manual intervention.
  • New releases from tool vendors are supported almost immediately. Tasktop’s test lab catches API changes as soon as a SaaS release is rolled out.
  • Tasktop’s change detection algorithm does not overburden endpoint systems; performance is not gated by the number of artifacts synched.
  • Tasktop can handle simultaneous updates and conflicts resulting from differences in workflow rules among endpoint systems.
  • Operates even when endpoints are disconnected or offline.
  • Can be used with on-premise, on-demand or hybrid infrastructures
  • Tasktop’s integrations are bi-directional, easy to deploy and configure.

For more information, please do not hesitate to send your inquiries to info@tasktop.com, or post your questions and comments below.

Categories: Companies

Product Owner Certifications

Scrum Expert - Mon, 06/20/2016 - 16:30
There are three roles in a Scrum team: the ScrumMaster, the developer and the product owner. This last role might be the most important for a team that uses an Agile project management approach in software development. The product owner has the responsibility of building and prioritizing the backlog. If this is not done properly, you will just not build the right product. This article presents the certifications available for product owners in Scrum. Author: Franco Martinig, Methods & Tools, http://www.methodsandtools.com/ Updates June 20 2016: added IEEE & SAFe certifications Certifications for product owners in Scrum are mainly provided by the two main Scrum organizations: the Scrum Alliance and Scrum.org. There has been also the creation of certifications provided by other organization like the IEEE Computer Society or the Scaled Agile Framework (SAFe). The training for these certifications is available from training organizations that are accredited by these two certification bodies. Certified Scrum Product Owner by the Scrum Alliance The Certified Scrum Product Owner (CSPO) is a certification provided by the Scrum Alliance. To be certified, you will need to attend a two-day CSPO training course taught by a Certified Scrum Trainer. In this course you will learn how Scrum works and the role of the Product Owner in a Scrum team. The topics covered in this course include managing stakeholders, ROI, grooming the requirements backlog, writing effective user stories, defining acceptance criteria for user stories and the definition of “Done.” Find more information on http://www.scrumalliance.org/certifications/practitioners/cspo-certification Professional Scrum Product Owner [...]
Categories: Communities

Audacious Salon at Agile2016 Conference

Scrum Expert - Mon, 06/20/2016 - 16:25
Every year Agile Alliance challenges the Program Committee to come up with something new, different, and a bit experimental. This year Agile2016 track chairs George Dinwiddie and Doc Norton offer 14 sessions in the Audacious Salon track modeled after salons of the 1700s in France. The Audacious Salon Starting in the late 16th century and blossoming in the 17th and 18th, the salon was a major influence in the development of French culture, art, science, and politics. The Audacious Salon track reprises this embrace of the unknown and the different. The Audacious Salon is a place where strongly-held ideas can be discussed in civility so dialog can lead us places we have not yet dreamed. It’s a place to compare experiences and expand our own with the richness of others. A place to offer insights and hear how they fit for others as we hear the insights of others and test our own beliefs against them. These sessions will encourage bold and daring dialogs that give rise to new collaborations with peers, perhaps unlikely peers, and lead to a future of exploration in fruitful directions. Come to Audacious Salon sessions ready to discuss brand new ideas brought forward by highly-experienced experts in several fields. Expect active participation, conflict, and passionate conversations that continue outside of the room and beyond the conference. Speakers for this track will not be revealed because it is the conversation that is the star! For more information, visit https://www.agilealliance.org/agile2016/program/tracks/#audacious_salon
Categories: Communities

Article Review: Beyond the agility: Lean

Learn more about transforming people, process and culture with the Real Agility Program

BERTEIG’s expert in Kanban, Travis Birch, introduced me to Kanban through this link. It may be one of the most reputable sources for Lean/Kanban content online. One article I find particularly appealing is Beyond the agility: Lean. I’ll admit that one of the reasons I became hooked is because the phrase “Anybody who thinks we can overcome an emotional resistance with logic was probably never married. We can only overcome emotion with a stronger emotion.” Having been married, this peaked my interest. The rest of the article goes on to give a fantastic introduction into the agility, Lean, Kanban relationship and it served to deepen my understanding of all three. Great read!

 

Learn more about our Scrum and Agile training sessions on WorldMindware.comPlease share!
Facebooktwittergoogle_plusredditpinterestlinkedinmail

The post Article Review: Beyond the agility: Lean appeared first on Agile Advice.

Categories: Blogs

Unix: Find all text below string in a file

Mark Needham - Sun, 06/19/2016 - 10:36

I recently wanted to parse some text out of a bunch of files so that I could do some sentiment analysis on it. Luckily the text I want is at the end of the file and doesn’t have anything after it but there is text before it that I want to get rid.

The files look like this:

# text I don't care about
 
= Heading of the bit I care about
 
# text I care about

In other words I want to find the line that contains the Heading and then get all the text after that point.

I figured sed was the tool for the job but my knowledge of the syntax was a bit rusty. Luckily this post served as a refresher.

Effectively what we want to do is delete from the beginning of the file up until the line after the heading. We can do this with the following command:

$ cat /tmp/foo.txt 
# text I don't care about
 
= Heading of the bit I care about
 
# text I care about
$ cat /tmp/foo.txt | sed '1,/Heading of the bit I care about/d'
 
# text I care about

That still leaves an extra empty line after the heading which is a bit annoying but easy enough to get rid of by passing another command to sed that strips empty lines:

$ cat /tmp/foo.txt | sed -e '1,/Heading of the bit I care about/d' -e '/^\s*$/d'
# text I care about

The only difference here is that we’re now passing the ‘-e’ flag to allow us to specify multiple commands. If we just pass them sequentially then the 2nd one will be interpreted as the name of a file.

Categories: Blogs

Unix: Split string using separator

Mark Needham - Sun, 06/19/2016 - 09:22

I recently found myself needing to iterate over a bunch of ‘/’ separated strings on the command line and extract just the text after the last ‘/’.

e.g. an example of one of the strings

A/B/C

I wanted to write some code that could split on ‘/’ and then pick the 3rd item in the resulting collection.

One way of doing this is to echo the string and then pipe it through cut:

$ string="A/B/C"
$ echo ${string} | cut -d"/" -f3
C

or awk:

$ echo ${string} | awk -F"/" '{ print $3}'
C

I don’t like having to echo the string – it feels a bit odd so I wanted to see if there was a way to do the parsing more ‘inline’.

I came across this post which explains how to change the internal field separator (IFS) on the shell and then parse the string into an array using read. I gave it a try:

$ IFS="/" read -ra ADDR <<< "${string}"; echo ${ADDR[2]}
C

Works! We can even refer to the last item in the array using -1 instead of it’s absolute position:

$ IFS="/" read -ra ADDR <<< "${string}"; echo ${ADDR[-1]}
C

I’d not come across this use of the ‘read’ function before. The key is the ‘-a’ parameter. From the man page:

-a aname
The words are assigned to sequential indices of the array variable aname,
starting at 0. All elements are removed from aname before the assignment.
Other name arguments are ignored.

So we’re resetting the internal field separator and then reading the string into another variable as an array split on the ‘/’ character.

Pretty neat although now it’s longer than the original command and I’m sure I’ll forget the syntax.

Further down the page is another suggestion which seems even harder to remember but is much shorter:

$ echo ${string##*/} 
C

This drops from the beginning of the string up until the last occurrence of ‘/’ which is exactly what we want.

This way is the nicest and doesn’t require any echoing if we just want to assign the result to a variable. The echo is only used here to see the output.

Categories: Blogs

Links for 2016-06-18 [del.icio.us]

Zachariah Young - Sun, 06/19/2016 - 09:00
Categories: Blogs

Python: Regex – matching foreign characters/unicode letters

Mark Needham - Sat, 06/18/2016 - 09:38

I’ve been back in the land of screen scrapping this week extracting data from the Game of Thrones wiki and needed to write a regular expression to pull out characters and actors.

Here are some examples of the format of the data:

Peter Dinklage as Tyrion Lannister
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
Filip Lozić as Young Nobleman
Morgan C. Jones as a Braavosi captain
Adewale Akinnuoye-Agbaje as Malko

So the pattern is:

<actor> as <character>

optionally followed by some other text that we’re not interested in.

The output I want to get is:

Peter Dinklage, Tyrion Lannister
Daniel Naprous, Oznak zo Pahl
Filip Lozić, Young Nobleman
Morgan C. Jones, a Braavosi captain
Adewale Akinnuoye-Agbaje, Malko

I started using the ‘split’ command on the word ‘as’ but that broke down when I realised some of the characters had the letters ‘as’ in the middle of their name. So regex it is!

This was my first attempt:

import re
 
strings = [
    "Peter Dinklage as Tyrion Lannister",
    "Filip Lozić as Young Nobleman",
    "Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)",
    "Morgan C. Jones as a Braavosi captain",
    "Adewale Akinnuoye-Agbaje as Malko"
]
 
regex = "([A-Za-z\-'\. ]*) as ([A-Za-z\-'\. ]*)"
 
for string in strings:
    print string
    match = re.match( regex, string)
    if match is not None:
        print match.groups()
    else:
        print "FAIL"
	print ""
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
 
Filip Lozić as Young Nobleman
FAIL
 
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous', 'Oznak zo Pahl')
 
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
 
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

It works for 4 of the 5 scenarios but now for Filip Lozić. The ‘ć’ character causes the issue so we need to be able to match foreign characters which the current charset I defined in the regex doesn’t capture.

I came across this Stack Overflow post which said that in some regex libraries you can use ‘\p{L}’ to match all letters. I gave that a try:

regex = "([\p{L}\-'\. ]*) as ([\p{L}\-'\. ]*)"

And then re-ran the script:

Peter Dinklage as Tyrion Lannister
FAIL
 
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
FAIL
 
Filip Lozić as Young Nobleman
FAIL
 
Morgan C. Jones as a Braavosi captain
FAIL
 
Adewale Akinnuoye-Agbaje as Malko
FAIL

Hmmm, not sure if I did it wrong or if that isn’t available in Python. I’ll assume the latter but feel free to correct me in the comments and I’ll update the post.

I went search again and found this post which suggested another approach:

You can construct a new character class:

[^\W\d_]

instead of \w. Translated into English, it means “Any character that is not a non-alphanumeric character ([^\W] is the same as \w), but that is also not a digit and not an underscore”.

Let’s try plugging that in:

regex = "([A-Za-z\-'\.^\W\d_ ]*) as ([A-Za-z\-'\.^\W\d_ ]*)"
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
 
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous as Oznak zo Pahl(credited', 'Stunt Performer)')
 
Filip Lozić as Young Nobleman
('Filip Lozi\xc4\x87', 'Young Nobleman')
 
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
 
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

So that’s fixed Filip but now Daniel Naprous is being incorrectly parsed.

For Attempt #4 I decided to try excluding what I don’t want instead:

regex = "([^0-9\(]*) as ([^0-9\(]*)"
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
 
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous', 'Oznak zo Pahl')
 
Filip Lozić as Young Nobleman
('Filip Lozi\xc4\x87', 'Young Nobleman')
 
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
 
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

That does the job but has exposed my lack of regex skillz. If you know a better way let me know in the comments.

Categories: Blogs

Stop Calling It Theft: Thoughts on TheDAO

Radyology - Ben Rady - Sat, 06/18/2016 - 01:41
Like many people involved in Ethereum, my attention has been thoroughly captured by the recent events surrounding TheDAO. As an Ethereum miner, I have a little stake in this game. The reentrancy vulnerability found in TheDAO smart contract has resulted... Ben Rady
Categories: Blogs

Filtering objects to Optionals

Xebia Blog - Fri, 06/17/2016 - 17:36
A while ago I stumbled upon a Blog post by Natascha the Robot about Configuring a Constant Using Shorthand Argument Names in Swift. Which by itself is a great post, but I was most inspired by the Then library mentioned at the end of her post. Seeing how such a small amount of code could change the way we configure
Categories: Companies

Why Comparing Different Approaches Is Good

NetObjectives - Fri, 06/17/2016 - 16:59
Early in school, we all learned to compare and contrast things and ideas. It is what helps us learn and to understand and to perform better. So why do some software consultants seem to think it is wrong to compare and contrast approaches to software development? Why should we expected to be supportive of them all? They are not all equally good or appropriate. This is especially true of Lean and...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]
Categories: Companies

Uke the World

Portia Tung - Selfish Programming - Fri, 06/17/2016 - 13:44

The Lost and Foundlings

“Without music, life would be a mistake” – Nietzsche

When I was little, I was taken to Yamaha piano classes every Saturday. I remember sitting between two mini world talents as I banged at the keys, vaguely wondering how the other children managed to translate those tadpole scribbles into beautiful tunes.

Then one auspicious day, my mum asked, “Would you like to stop going to piano classes?” To which I replied, “Yes!” And that was one of my earliest memories of the sweet taste of liberation. And many other things besides.

The conclusion from this experience was stored in perpetuity in my brain’s computer. Because my computer was efficient, it readily replayed its recorded message at every opportunity: “I am not musically talented. I will never play a musical instrument. I cannot sing…”

… Until the day came when I became a parent and learned that, as a parent,”You can only give what you have”. Suddenly, from across the misty distance of time and space, I heard the familiar sound of banging keys and I was paralysed with fear. Of not being good enough. And this time, my child’s fate is at stake.

Then somewhere from deep within me, amidst my chilling and profound sense of loss, a gentle and defiant voice spoke up. “Wouldn’t it be nice to make my own music? To sing and play with all my heart?”

Without knowing how or when that wish would come true, the most important thing had happened, a pre-requisite for all wishes before they can come true: my heart had made a wish.

Within days, my friend Vira offered me the gift of music in the form of the ukulele so that I could share music with my little girl. And what better way to celebrate life than by sharing the gift of song?

Play for Your Life

I hope you’ll be able to join our uke jam keynote (our uke troupe’s called The Lost and Foundlings) at BCS SPA on Tuesday, 28 June where we’ll be supported by The Fleas who’ll bring along ukuleles so you, too, can have a go. No previous experience required, just the willingness to play!

And if you already play an instrument, do bring it to the conference with you that day because uke jams sound even better with all sorts of different instruments!

And if you’ve always had a longing to learn to play the uke and you fancy buying one to bring along, then a decent starter uke costs around £20-£30 (we recommend Kala or Mahalo available from Amazon and good music shops). Let the music play!

Categories: Blogs

Signing Bonuses: Indentured Servitude of the Engineering World

Recently I was having lunch with several friends, two of who worked for a certain Fortune 500 company. These two in particular were discussing a lot of negative situations at their current company and I asked the obvious question… “Why don’t you just leave?” I mean right now in my opinion it is definitely a software engineer’s market. Off hand I know of at least a dozen companies with the number of open positions in the double digits that they’re desperately seeking to fill. And while I know this is anecdotal, my inbox get a ridiculous amount of emails per day from recruiters who aren’t just asking me if I’d be interested, they’re practically begging me to please forward them resumes of anyone I know who might be a good fit. So with such booming opportunities, why put up with shitty behavior of an employer?

The answers I got were interesting. While one guy had the common excuse I see where they believe their employer, despite being crummy, is the only well paying opportunity available the other gentlemen told me something I was rather surprised to hear. When he joined he was given a $10,000 signing bonus. He happily used this to pay for a bunch of of home improvements. But this wasn’t just free money, it came with some strings attached. I might have the details muddy but from what I heard the sum had to be returned in full (even including the amount that got taxed out of the original $10,000) if the employee terminates employment with the employer within 2 years. Basically what it comes down to is he has to pay $10,000 to leave and even if someone hadn’t spent the money they still need to scrap up the amount taxed on it in order to leave. What!?

I’ve heard of signing bonuses all the time and a lot of companies offer some pretty sweet stock options to prospective employees with the typical constraints (e.g. they take 3 years to vest fully) but this is the first time I heard of a signing bonus that has to be returned if the employee doesn’t complete two full years of employment. With so many of the expenses people have from mortgages to car payments to child care scraping up that amount is no small task and might even be impossible. As with my friend here he’s pretty much stuck with an employer he doesn’t like for the next year because there’s no other option available to him aside from taking out a loan to pay the signing bonus back. That’s no signing bonus… that’s indentured servitude.

The point to really ponder when you’re mulling over several prospective job offers is to understand that anything that is a bonus for signing on is just that, a bonus. Never take a job based on any signing bonus they offer. Stock options are great but don’t accept them in lieu of the full market value for your abilities. And if any company comes along with a signing bonus of a lump sum be sure you read the fine print. If it comes with a constraint where you must pay it back run straight for the hills and make sure you let them know that you won’t work for indentured servitude. Don’t sell yourself short! There’s tons of software engineering jobs out there and a ton of them that don’t come with such ridiculous, almost outright vindictive strings attached!

Categories: Blogs

Knowledge Sharing


SpiraTeam is a agile application lifecycle management (ALM) system designed specifically for methodologies such as scrum, XP and Kanban.