Skip to content

Agile Estimator
Syndicate content
Controlling the Chaos of Agile Development
Updated: 44 min 31 sec ago

Draft Scope and Boundary

Thu, 05/18/2017 - 23:14

Size Distribution of New Development Projects

From the very beginning, all participants should start to form some expectation regarding the size of the application to be estimated. At the top of the post is the size distribution of new development projects. The size distribution is expressed in IFPUG function points. The methodology presented here uses IFPUG sizing measures. At this point, there is not enough industry experience to present a similar distribution for SNAP points. Even if there was, function points would still be the best way to express the size of an application that is about to be built. SNAP points become more important when estimating the schedule and effort for projects that have changes in functionality, such as enhancement projects. In any case, The sizing methodology presented here will take into account both function points and SNAP points.

if an applications is 50 function points or less, it should simply be assigned to an agile team and they should handle any estimating and reporting. The initial team size should be based on management’s experience with similar applications. Team size can be adjusted in future iterations.  At the other extreme, only 10% of the projects studied had function point counts that exceeded 2,000. There is simply not enough experience with these larger applications to proceed without extra caution. When an estimator is convinced that a project is more than 2,000 function points in size, the possibility of dividing the project into separate components should be considered. Otherwise, the process can still be used, but additional care must be taken to insure functionality is not being missed. These huge projects can almost never be done by a single agile team.

According to industry practice, establishing scope and boundary is a must for any estimating exercise. Since this process is basically function/SNAP point based, it made sense to use the standard IFPUG definitions for these attributes. Scope and boundary rules are basically the same for both function points and SNAP points. The first step it to draft the scope and boundary of the application to be estimated. The scope and boundary may change during the estimating process. This may be the result of better understanding of the project or a change in scope. When this happens, the estimating process has to be reworked.

Prior to defining the scope and boundary, the purpose and type of assessment must be determined. This process was developed with a certain purpose and type in mind. The purpose of any assessment is to estimate the size of an application to be developed so that the size can be fed into an estimating model. The size will include both functional and non-functional requirements. All agile application development efforts involve taking some software and converting it into some other software with additional or changed capabilities. This type of assessment is an enhancement project. New development can be treated as a special case of enhancement. In new development, there is no existing capability to begin with. In days of old, the waterfall methodologies favored this special case of new development. You began with nothing and continually transformed models until you ended up with a completed application. For example, you did analysis and ended up with a data flow diagram of the entire system to be built. Then you transformed that into a structure chart to show all the components to be built. Then you coded the design to have a system. That was classic new development. In agile development, you are continually transforming working software from one form into another. A simple release is built that contains a workable subset of requirements. Then, the attributes of that release are changed and augmented to produce the next release of the software. It is basically software development by continual enhancement. Fortunately, this approach also works if the team is tasked with enhancing an existing piece of software that they did not build.

According to the IFPUG Counting Practices Manual (CPM), the scope “defines a subset of the software being sized.”  It identifies which functions will be included in the size so as to provide the best estimate. A project might only be enhancing certain portions of a system. There may be reusable components that will supply some of the functionality to be delivered. All of this must be considered in the scope. For people using the IFPUG sizing methodologies, sizing can be complicated. For people using the process outlined here, it is fairly straight forward. The scope includes any existing software attributes that will be changed and any new attributes that will be added. People familiar with IFPUG methodologies may ask about deleted attributes. Most estimating models ignore deleted attributes. They are ignored here.

The CPM states that the boundary “defines what is external to the application.” If the application being built must interact with other applications, then these applications are outside of the boundary of the one being developed. People who use the application are outside the boundary of the application. The bad news is that establishing boundaries can be complicated. For example, is there a boundary around Microsoft Office? Or, is the boundary around Excel, Word , PowerPoint, etc.? How about QuickBooks? It seems like there should be a single boundary around QuickBooks. However, there are several versions of QuickBooks. There are several for the PC that have extended functionality. There is one for the Mac. There is a version that only works on the web. Some of them probably share portions of each other’s code. Where does the boundary get drawn? Fortunately, there are two pieces of good news. The first is that when using this process, the boundary is usually drawn around the attributes of the application that will be changed and includes the things that will be added. Sometimes, there will be changes made to multiple applications. In that case a boundary will be drawn around each. Who decides and how do they do it. This is the second piece of good news. It is the users, and probably the product owner, that is most qualified to make this determination. They make it by deciding how users view the software they are using. For example, do they think they are using Microsoft Office or Microsoft Excel? It is probably the latter and that will drive their boundary decisions.

The scope and boundary may emerge from a study of the initial stories that define an application. Otherwise, they must be established by some other collaboration between the estimator(s) and the stake holder(s) of the application. The definitions of scope and boundary for this process are completely consistent with those found it the IFPUG CPM. Of course, these definitions of scope and boundary are probably generic enough to be used with virtually any sizing measure.

People who are familiar with the SNAP process may be surprised that no mention has been made of partitions. Partitions are identified as part of this process. However, they are not identified at this time.

Categories: Blogs

Counting Batch Processes

Fri, 05/05/2017 - 05:13

Counting Batch Processes

It seems like batch processing should have no place in agile development. It conjures up the image of someone sitting at a IBM 026 keypunch machine typing in Job Control Language (JCL) and COBOL II. It is doubtful that any agile team has ever experienced this. In any case, that is not what is being counted here.

Batch processes are non-functional processes that do not cause information to cross the application boundary. A typical example of that would be some type of end of month processing. For example, there might be a program that changes the running totals that are maintained in a file. There might be 12 running totals for the past 12 months. The program might discard the oldest total and move the other totals between the 11 months that remain. Then, a new total might be placed in the 12th bucket. Nothing has crossed the application boundary. The program runs because it is scheduled to run at the end of the month. Likewise, it does not issue any kind of user report. End of day, week, month, etc. are not the only possible batch processes. Programs that load information into files may also be batch processes. This is only the case if they have not been counted as conversion functionality.

Sometimes, it is easier to identify batch processes by describing what they are not. Batch processes are not functional. Therefore, anything that is functional is not counted here. If data is transferred to or from an external application, then it is probably functional unless it is updating a code table. Anything reading screens or generating reports are probably functional. In any case, they are not batch processes. When data in an internal logical file (ILF) is updated as part of an enhancement, it is usually considered conversion functionality. For example, if a new field is added to an ILF to contain someone’s birthday, then part of the project would be the conversion job to fill the birthday fields of existing records. What if there were a one-time job to update birthdays from another source. If the birthday field were not new, then the process would not be conversion. However, many practitioners have counted these one-time jobs as conversions. If so, then they cannot be counted as batch processes.

Not all non-functional processes are batch processes. Anything that is designed to be performance enhancing is not considered to be a batch process. Changes to the data in code tables are non-functional, but they are not batch processes. These processes are covered by other non-functional categories. Any batch processes that have not been counted by one of these other functional or non-functional processes can be considered a batch process.

How do you group batch processes? This is where your JCL experience would be helpful. Batch processing was broken into jobs. Jobs were broken into steps. For estimating purposes, consider the job level. All of the steps that occur for that job would be considered part of the batch process.

When calculating the SNAP points for a batch process, first count up the number of File Types Referenced (FTR). If a batch process must read from a customer file, then the FTR number is one. If multiple steps of the batch process read from different files, then the FTR number is the sum of the unique files. In other words, if two steps read from the customer file, the FTR is still one. If one step reads from the customer file and the next writes to the order file, then the FTR number is two. The number of FTRs is used to establish the complexity, as shown in the table at the top of this post. If there is three or less FTRs, there is low complexity; five to nine FTRs, average complexity; 10 or more means high complexity.

The SNAP count is a function of the complexity and the number of Data Element Types. If the complexity is low, the SNAP count is four times the number of DETs; average, 6 times; high, 10 times. This is a time where the DET count must be arrived at with care. Many times in sizing, the DET count becomes part of a range. For example, when arriving at a function point count for an External Input, any DET count over 15 becomes high complexity. It does not matter whether the count is 16 or 60. The situation is different with the SNAP count for a batch process. If an average complexity batch process works with 16 DETs, the SNAP count is 96; for 60 it would be 360. Therefore, an accurate count or estimate of the DETs is necessary. Many times estimators use the number of fields as a DET count. That is not true. Using the example that was mentioned earlier, suppose that the batch process was updating 12 rolling totals. This is one DET, not 12. The repeating groups are all the same DET. There are some borderline cases. Is the date one DET or 3 (month, day and year)? It depends upon whether the different date components are used differently. The estimator should use their judgement.

Here is our example: if there is an end of month process that discards the oldest running total, shifts each of the 11 running totals down one, and puts a new running total in the last bucket, then it is a batch process. Since it only interacts with a single running total file, its FTR is one and the batch process has low complexity. Since it has low complexity and a single DET, the SNAP count is 4 (4 x 1).

The one question on people’s minds should be, “How can I estimate the number of DETs early in the life cycle?” A better question might be, “Can I identify the batch processes early in the life cycle?” The bad news is that you can not. These are the types of things that are identified in the design phase. Did I get your attention? There is no design phase in an agile development effort. There are no analysts, designers or testers. There is only development being done by developers. Developers are clarifying requirements from the user stories during the iteration where they are being implemented. Developers are designing the simplest solutions that work during each iteration and then refactoring those designs when it makes sense to do so. However, all of this development is done during iterations (sprints). The need for batch processes is usually uncovered during later iterations. It is usually after files have been put into place and the need for maintaining different types of data have been uncovered. The good news is that the batch processes do not impact estimates early in the life cycle as much as later. Early iterations are often dominated by additions to functionality. Later iterations and enhancements may be dominated by changes to non-functional aspects of the system.

There are other reasons to identify batch processes. Suppose there is a sprint where it is anticipated that 5 user stories will be implemented. While working on the stories, the need for some bath processes arises. Implementing those processes will probably result in a story or two not being implemented. This is the result in the expansion of software size, not a drop in velocity. This has to be understood so as not to underestimate velocity on future iterations.

People who are using estimating models like COCOMO II, SEER or SLIM also have to capture the programming language or languages being used to implement the batch process. If there are multiple languages, then they must be apportioned. For example, if 50% of the batch process is written in JavaScript and the other 50% is in Perl, then this should be captured.

Categories: Blogs