A project manager is equipped with metrics, cost models, and project audits not only to solve current project-related problems but also to improve performance of future projects.
Management metrics measure how closely a project conforms to development plans and assess whether an acquisition is at risk of delay or cost increases. They provide quantitative information.
These techniques require considerable information about the
system being developed and can be used only after system design. The metrics will generally be used after contract award, in order to assess how well the organization is managing the system development process.
Software metrics, which use mathematical models to measure elements of the development process, are intended to help organizations better understand and manage the relationships between resource decisions, development schedules, and the cost of software projects.
By using software metric tools, an auditor can independently evaluate software development projects by analyzing project budgets, requirements, schedules, and resources, and then confirming or questioning cost, schedule, and resource estimates.
Different metrics may be useful for an audit, depending on the objectives and status of an acquisition.
Using cost models to estimate the cost and length of time necessary to develop a new software, for example, is appropriate only after requirements have been defined, a system design has been developed, and the size of the new system has been estimated. The models described here require estimates of either the lines of code or number of function points that the new system will include.
Cost models project the time and cost to develop a system on the basis of estimates of the system’s size and other pertinent factors. Cost models may be used before a solicitation for software development is issued in order to assess the reasonableness of the project schedule (if a system design has been prepared) or after a contract has been awarded and software development has begun.
Cost Models
Cost models are tools that estimate the effort needed to develop software, based on assumed relationships between the size of a system and the effort needed to design, code, and test the software.
These models can help the auditor assess whether the acquisition’s estimated cost and schedule are reasonable. Each model uses cost drivers, which are parameters such as the level of experience of the programmers, the reliability requirements of the programs, and the complexity of the project, along with the estimated project size, to derive overall cost and schedule estimates for the acquisition.
When a system involving software is being developed, one or more cost models may be useful. A model will be more reliable if it takes into account the organization’s historical experience in developing computer based information systems. Typically cost models are derived using data gathered from years of experience with a wide range of software development projects.
Many cost models are based on industry-recognized models, such as the Constructive Cost Model (COCOMO), PRICE, Putnam, and Jensen.
Auditors should use these cost models to look for discrepancies with the cost and schedule estimates established for an acquisition. By varying the estimated values for cost drivers, the auditor may also be able to perform a sensitivity analysis illustrating where project estimates are most susceptible to change.
Limitations of Cost Models
However, cost models have significant limitations in accuracy unless their underlying assumptions of system size and cost drivers are carefully chosen and reflect the organization’s previous experience in system development.
It is a matter of auditor judgment to decide how discrepant project estimates and estimates provided by cost models should be to raise concerns about risks of cost and schedule overruns. In making this judgment, the project auditor should take into account the uncertainty of estimates and assumptions made in using a cost model.
Project auditors should use a cost model to provide general estimates and not precise figures. This is due to the nature of estimates in that they are based on projections of an inherently uncertain future. Auditors should also ensure that the cost model used is consistent with the software development methodology of the project under consideration.
A system developed through rapid prototyping, for example, should be evaluated with a cost model that takes prototyping into account.
Other Project Indicators
In addition to cost models, five indicators such as problem reports, software volatility, development progress, software size, and staff stability are important to assess how effectively an organization is managing system development projects (in-house or contractor developed).
These indicators measure differences between what an organization planned for its contractor to accomplish by certain points in the system development life cycle and the actual results.
In most cases, this comparison is most easily done graphically. Using more than one indicator can give a broader picture of a project’s status. Auditors should use indicators appropriate to the project under review and for which data are available.
These indicators require that the system development work be underway. A comparison of the actual number of functions successfully tested to the number planned in the acquisition schedule, for example, requires that system testing be underway.
https://www.scoro.com/blog/16-essential-project-kpis/
Their main difficulties arise from three sources:
The working relationships between the project manager and functional department managers have not been clearly defined by senior management.
Uncertainties arise with respect to handling delays, cost overruns, work assignments, and design changes. Unless the project manager is skillful in handling these situations, senior management may resolve them in the interest of functional departments, at the expense of the project as a whole.
When uncertainties are added to the situation, the project manager has to make his decisions based on limited data and with little or no analysis. He must move fast, even if it means an intuitive decision that might expose him to senior management’s criticism.
Decisions to sacrifice time for cost, cost for quality, or quality for time, are common in most projects. There is a clear indication that the project manager needs support from senior management due to these trade-offs.
Senior management can seldom give the project manager as much guidance and support as his line counterpart gets.
Delays in initial approval of the project by senior management, inability to resolve conflicts between the project manager and the functional department managers, and delays in allocating resources are the most common issues on which the project manager needs more attention from senior management. Otherwise, project performance can be hampered.