W16_SJP_Accuracy, Precision and Reliability

Issue Identification and Appraisal

Blog 15 discussed the tracking and reporting Cost Performance Index (CPI) and Schedule Performance Index (SPI) and the graphical choices, which brings out another dimension – the results and how close they are to the baseline, or was the baseline flawed. This week’s blog problem statement is, “With the CPI/SPI results obtained to week 15, what classification was the original baseline?”

Feasible Alternatives

Review of the Guild of Project Controls Compendium and Reference (GPC) Module 9.4 ‘Assessing & Interpreting Progress Data’ provides three feasible alternatives.

These are:

  • Accuracy
  • Precision
  • Reliable

Develop the Outcomes for each

The GPCCAR Module 9.4 figure 9, demonstrates that there are several options regarding the feasible alternatives, and that a combination of two or better still all three would be beneficial.

Figure 1 – Precise

Figure 1 shows that the CPI/SPI points are consistently ‘hitting’ the target and within the acceptable range, which demonstrates that the baseline estimate was Accurate, Precise and Reliable to be used in any future projections.

Figure 2 – Imprecise

Figure 2 shows that the CPI/SPI points are scattered over the target and therefore imprecise, which demonstrates that the baseline estimate was not Accurate or Reliable and should not be used in any future projections.

Figure 3 – Precise and Accurate with some outliers

Figure 3 shows that many the CPI/SPI points are ‘hitting’ the target but a few of the outlying points need to be assessed as to why they were outside. In such cases the Project Controls professional could apply statistical process control chart analysis to discard these outlying points that are outside of the +/- 3 Sigma range as being “special and identifiable”. Use of these values without performing any statistical process control would be risky.

Figure 4 – Precise and Reliable but not Accurate

Figure 4 shows that the CPI/SPI points are ‘hitting’ the target in a precise group but outside of the bulls-eye. It shows that the data is precise and reliable, but needs to be adjusted to bring the points back to the center. If adjustments were made, then the data could be used in future projections. Adjustments would mean looking at the cost and time estimates and assessing what needs to be done to gain alignment.

Table 1: Summary of Data Point Dispersions

Selection Criteria

The criteria selection for both models will be taken from W05 through W15 weekly reports. Table 2 below shows the information to be used.

Table 2: CPI & SPI Data points for use in Analysis

Analysis and Comparison of the Alternatives

The data points from table 2, were input to the “Target” Cost/Schedule Trend chart to provide the following graphical output.

Figure 5: Cost/Schedule Trend Report

Review of the data, shows four data-points outside of the +/- Sigma 3 range. As mentioned in the figure 3 explanation these four points needs to be assessed to determine why they fell outside to ensure there is a valid explanation to discard them.

Figure 6: Zoom-in View

To understand the reasons for the outlying points we need to determine which week these occurred and check the report analysis for explanations.

Table 3: CPI & SPI Data points outside +/- 3 Sigma limits

Table 3 shows the four data-points that need reviewed to determine why they fell outside the limits. Of the four points, three are schedule related only, and one is to cost and schedule.

The first was in W06 and was due to missing final 3 activities in the Paper 1. In W08 the trend had reversed and schedule and cost were showing they were well ahead. This was due to the budget being inflated, this was subsequently revised in W09. In W10 schedule was showing it was ahead but due to the papers going better than anticipated because of working smart, and not following the template set out by Dr. Paul Giammalvo. W13 the schedule was back on the radar as paper 4 was progressing good due to research being done on blogs prior to commencing the paper thus reducing the development time.

Figure 7: Data Points without Outlying points

Therefore, our initial scenario was like the “Precise and Accurate, with Outliers. Once these outlying data points are discarded the other points are dispersed within the +/- Sigma 3 ranges.

Selection of Preferred Alternative

To provide a level of confidence in the data points, a statistical control chart will be used. As this requires one set of data points and a time scale, the data points will be based on CPI * SPI.

Table 4: CPI * SPI Data points for Control chart

Inputting the above points into the statistical control chart model provides the output in figure 8. Looking at the chart it tells us the following:

  • Most points are close to the average
  • There are a few points close to the limits
  • There are two points beyond the control limits

Figure 8 : CPI*SPI Statistical Control Chart

As there are a couple of points outside of the control limits these are treated as a “special cause”. These occurred prior to the budget re-alignment, looking at figure 8 it shows that points from W09 onwards are in control and therefore generally acceptable.

With regards the Control budget, it appears it is both precise and accurate and can be used in evaluating future forecasts based on the data points provided.

Monitoring Post Evaluation Performance

As is the case with the other blogs, close monitoring of the period data allows regular evaluations to ensure that the data-point trends are keeping within the Statistical Control limits. If these limits are breached then re-evaluation of the data will be required.



One Reply to “W16_SJP_Accuracy, Precision and Reliability”

  1. You did a really great job up until Figure 8. While you correctly identified the outliers that were caused by “special” or “identifiable causes, what I was hoping you would do is also look for PATTERNS within the upper and lower control limits that would indicate something is wrong with the PROCESS itself?

    Also I was disappointed with Figure 8 that you did not differentiate between the Upper and Lower CONTROL limits and the Upper and Lower SPECIFICATIONS limits?

    As this topic is definitely in the exam question bank, I would urge you for your W17 blot would be to:
    Look at the SPI and CPI separately and analyze against both the Upper/Lower CONTROL limits and the Upper and Lower SPECIFICATION limits.

    I would also urge you not only to look for the SPECIAL or IDENTIFIABLE causes but also for any PATTERNS that would indicate something might be going wrong with the PROCESS itself? (As opposed to external events) I am really keen to see what your analysis tells you when you apply SPC tools/techniques.

    Dr. PDG, Jakarta


Leave a Reply

Your email address will not be published. Required fields are marked *