Chapter 2 - Project Evaluation
Summary:
This Chapter identifies key steps that should be outlined and evaluated before the commencement of a construction project, including: project concept; public interest; the role of the project team; and the likely costs, benefits and risks associated with the project. In addition, this Chapter addresses the likely environmental impact that a given project may cause and how proposed projects may incorporate sustainable practices and comply with state and federal regulations.
§ 2.1 Community Priorities
What does the public really want? What can they afford? Two simple questions, perhaps, but those rarely asked or carefully discussed when conceiving a major project. Is the local airport commission proposing an airport expansion — or should the county government really focus upon meeting regional transportation needs — of which an airport expansion might not be a useful element?
What is the universe of transportation alternatives? Which transportation hubs are going to be overly congested and become dysfunctional in the near horizon? How will these alternatives be affected by external forces, such as $10 per gallon gasoline, rising crime or bioterrorism?
Project authorizations should not occur in a vacuum — but should evolve within an evaluative framework of urgent and long-term community needs. A project should not get the green light based upon a compelling presentation by an articulate consultant (who can pitch any project with dynamic skill), or an architecturally stunning model that has been nicely rendered, digitally enhanced and cinematically projected against the wall of the council chambers. A prudent capital decision should not hinge on either the availability or “loss” of matching state or federal funds. It should be evaluated on its fundamental merits and true costs.
§ 2.2 The Project Concept
The project concept may come in a sudden vision of a community leader, a grass roots movement, the local business, education or sports community, a local governmental entity with a long standing plans for such a project, or be the product of dire necessity after a local disaster or catastrophic event. At times, a farsighted organization outside of the community may suggest a local project. Or, in the case of California’s prison and health system, the impetus for large capital projects may issue from the rulings of a United States Federal Judge directing the state to meet enhanced standards of habitability and health care.
As a practical matter, the need for a major regional project arises through a growing consensus of many disparate constituencies and will be successful if it has committed and widespread support.
In reality, the suggested airport expansion or sports arena might not be a real priority — or given the cost, even worth considering. Other projects may need civic focus and attention — such as youth softball parks, safe highways and emergency services centers. In other words, the shiny new project might not be a responsible public expenditure.
A local, elected public body is traditionally the most community-oriented, environmentally sensitive and budget conscious of political bodies. Yet, its members can be swayed by the project sponsors and consultants. The rock star architect may enjoy focusing on the “look and feel” and “project significance” (translation: “design awards”) of the initial sketches, but may not be concerned by project budget or schedule.
A good project engineer will advocate public safety, long-term durability and cost-efficiency. The successful contractor will fuss over local and qualified labor and subcontractors, the intricacies of the project schedule, the constructability of the plans and specifications, the financial opportunity, physical risk and likely profitability.
The ever-pragmatic gnomes, the ultimate facility users, whether ordinary citizens, school administrators or bridge operators worry they may be left holding the bag — one stuffed chock full of operational issues, affecting project suitability, practicality and long-term maintenance costs. Every participant covets their stripe of the project rainbow.
§ 2.3 Development of the Project Team
Who will organize and manage the project? Most owner organizations do not have sufficient depth of design and construction expertise to self-manage their own projects. So that expertise must be acquired by hiring or contracting with specialized firms.
Note: The structure and organization of the project team is covered in depth in Chapter 6.
§ 2.4 Evaluating the Likely Benefits
During the early planning stages of a project, the key goals and objectives are identified and prioritized. The conflicting goals of speed in delivery, ease of operation, dramatic design elements, innovation, national expertise, local content, proven technology and cost efficiency can rarely be reached simultaneously. Choices must be made at the highest levels of the client organization and will largely determine the success of the project.
§ 2.5 Estimating True Project Costs
There exists a body of literature and scholarship on estimating true and complete project costs and benefits that would fill a local public library to the top shelves. Yet, lifecycle costing is often overlooked during a public agency’s “Green Light” decision.
The U.S. Office of Management and Budget has issued the useful Circular A-94 for the purpose of articulating the overriding principles and guidelines for the evaluation of costs and benefits of U.S. Federal projects.
Note: This book provides a glossary of infrastructure terms used in this Circular; technical terms are italicized when they first appear.
The Circular provides this overview and analysis of the subject:
General Principles. Benefit-cost analysis is recommended as the technique to use in a formal economic analysis of government programs or projects. Cost-effectiveness analysis is a less comprehensive technique, but it can be appropriate when the benefits from competing alternatives are the same or where a policy decision has been made that the benefits must be provided.
a. Net Present Value and Related Outcome Measures. The standard criterion for deciding whether a government program can be justified on economic principles is net present value — the discounted monetized value of expected net benefits (i.e., benefits minus costs). Net present value is computed by assigning monetary values to benefits and costs, discounting future benefits and costs using an appropriate discount rate and subtracting the sum total of discounted costs from the sum total of discounted benefits. Discounting benefits and costs transforms gains and losses occurring in different time periods to a common unit of measurement. Programs with positive net present value increase social resources and are generally preferred. Programs with negative net present value should generally be avoided.
Although net present value is not always computable (and it does not usually reflect effects on income distribution), efforts to measure it can produce useful insights even when the monetary values of some benefits or costs cannot be determined. In these cases:
1. A comprehensive enumeration of the different types of benefits and costs, monetized or not, can be helpful in identifying the full range of program effects.
2. Quantifying benefits and costs is worthwhile, even when it is not feasible to assign monetary values; physical measurements may be possible and useful.
Other summary effectiveness measures can provide useful supplementary information to net present value, and analysts are encouraged to report them also. Examples include the number of injuries prevented per dollar of cost (both measured in present value terms) or a project’s internal rate of return.
b. Cost-Effectiveness Analysis. A program is cost-effective if, on the basis of life cycle cost analysis of competing alternatives, it is determined to have the lowest costs expressed in present value terms for a given amount of benefits.
Cost effectiveness analysis is appropriate whenever it is unnecessary or impractical to consider the dollar value of the benefits provided by the alternatives under consideration. This is the case whenever (i) each alternative has the same annual benefits expressed in monetary terms; or (ii) each alternative has the same annual affects, but dollar values cannot be assigned to their benefits. Analysis of alternative defense systems often falls in this category.
Cost-effectiveness analysis can also be used to compare programs with identical costs but differing benefits. In this case, the decision criterion is the discounted present value of benefits. The alternative program with the largest benefits would normally be favored.
c. Elements of Benefit-Cost or Cost-Effectiveness Analysis.
1. Policy Rationale. The rationale for the Government program being examined should be clearly stated in the analysis. Programs may be justified on efficiency grounds where they address market failure, such as public goods and externalities. They may also be justified where they improve the efficiency of the Government’s internal operations, such as cost-saving investments.
2. Explicit Assumptions. Analyses should be explicit about the underlying assumptions used to arrive at estimates of future benefits and costs. In the case of public health programs, for example, it may be necessary to make assumptions about the number of future beneficiaries, the intensity of service, and the rate of increase in medical prices. The analysis should include a statement of the assumptions, the rationale behind them, and a review of their strengths and weaknesses. Key data and results, such as year-by-year estimates of benefits and costs, should be reported to promote independent analysis and review.
3. Evaluation of Alternatives. Analyses should also consider alternative means of achieving program objectives by examining different program scales, different methods of provision, and different degrees of government involvement. For example, in evaluating a decision to acquire a capital asset, the analysis should generally consider: (i) doing nothing; (ii) direct purchase; (iii) upgrading, renovating, sharing, or converting existing government property; or (iv) leasing or contracting for services.
4. Verification. Retrospective studies to determine whether anticipated benefits and costs have been realized are potentially valuable. Such studies can be used to determine necessary corrections in existing programs, and to improve future estimates of benefits and costs in these programs or related ones. Agencies should have a plan for periodic, results-oriented evaluation of program effectiveness. They should also discuss the results of relevant evaluation studies when proposing reauthorizations or increased program funding.
5. Identifying and Measuring Benefits and Costs. Analyses should include comprehensive estimates of the expected benefits and costs to society based on established definitions and practices for program and policy evaluation. Social net benefits, and not the benefits and costs to the Federal Government, should be the basis for evaluating government programs or policies that have effects on private citizens or other levels of government. Social benefits and costs can differ from private benefits and costs as measured in the marketplace because of imperfections arising from: (i) external economies or diseconomies where actions by one party impose benefits or costs on other groups that are not compensated in the marketplace; (ii) monopoly power that distorts the relationship between marginal costs and market prices; and (iii) taxes or subsidies.
§ 2.6 Evaluation of Project Risk
What is the risk profile of the project? One-of-a-kind or once-in-a-generation facilities are prone to massive overruns and design problems due to agency inexperience. Airports, jails, sewage treatment facilities, transit systems, administration facilities and community centers are also susceptible to these problems. The nature of these risks should drive the project strategy used by the owner.
On the other hand, repetitive projects are regularly accomplished by experienced and well-financed public owners. K-12 schools, roadways, bridges, and sewer and water pipelines are typical examples of projects that should, generally, go right. But these also have risks, although less obvious. The evaluation and mitigation of project risks is explained in Chapter 3, below.
The OMB Circular 94-A, mentioned in Section 2.5, suggests the practical limits on predicting the frequency, severity, statistical distribution and resulting expected value of project uncertainties (items 9 & 10):
Treatment of Uncertainty. Estimates of benefits and costs are typically uncertain because of imprecision in both underlying data and modeling assumptions. Because such uncertainty is basic to many analyses, its effects should be analyzed and reported. Useful information in such a report would include the key sources of uncertainty; expected value estimates of outcomes; the sensitivity of results to important sources of uncertainty; and where possible, the probability distributions of benefits, costs and net benefits.
Characterizing Uncertainty. Analyses should attempt to characterize the sources and nature of uncertainty. Ideally, probability distributions of potential benefits, costs, and net benefits should be presented. It should be recognized that many phenomena that are treated as deterministic or certain are, in fact, uncertain. In analyzing uncertain data, objective estimates of probabilities should be used whenever possible. Market data, such as private insurance payments or interest rate differentials, may be useful in identifying and estimating relevant risks. Stochastic simulation methods can be useful for analyzing such phenomena and developing insights into the relevant probability distributions. In any case, the basis for the probability distribution assumptions should be reported. Any limitations of the analysis because of uncertainty or biases surrounding data or assumptions should be discussed.
While the cost impacts of many on-site construction risks can be mitigated through insurance, the most serious risks to public owners and contractors are delays and cost overruns. Neither of these calamities is generally insurable. Loss of life and serious injuries, while insurable to a degree, may involve losses that easily exceed the combined insurance coverage, as if money is ever a substitute for such grievous losses. So, risk management is at the heart of successful projects.
§ 2.7 Environmental Impact & Mitigation
In 1969, Earth Day launched a new era in environmental awareness. Whether spurred by the famous photo of “Earthrise” taken from the Apollo Spacecraft orbiting the moon or the resurgent energy of a nation grown tired of a gristly war in Southeast Asia, the sudden spread of public awareness of the uniqueness of human existence, on a crowded planet, resulted in an explosion of policy, legislation and regulation to protect the environment.
In fact, it was a Californian, President Richard M. Nixon, who in 1969 appointed the first Chairman of the Council on Environmental Quality and signed into law the National Environmental Policy Act (NEPA). And it was then-California Governor Ronald Reagan who signed CEQA, the California counter-part, into law in 1970 thus formalizing California agency environmental decisionmaking.
NEPA directs federal agencies to examine on a detailed level the environmental impact of proposed federal projects and study and implement mitigation measures to lessen the project’s impact on the environment. That statute was followed by state statutes of similar import, such as the aforementioned, California Environmental Quality Act (CEQA) in 1970.
As stated by California’s State Resources Agency, “… the California State Assembly created the Assembly Select Committee on Environmental Quality to study the possibility of supplementing NEPA through state law. This legislative committee, in 1970, issued a report entitled The Environmental Bill of Rights, which called for a California counterpart to NEPA. Later that same year, acting on the recommendations of the select committee, the legislature passed, and Governor Reagan signed, the CEQA statute.”
The Nixon era saw a “Cambrian” explosion of Federal environmental legislation including the National Clean Air Act (United States Code, Title 42, Chapter 85) of 1970, the Endangered Species Act of 1973 (ESA) (7 U.S.C. § 136, 16 U.S.C. § 1531 et seq.) and the Federal Water Pollution Act Amendments of 1972 (33 U.S.C. §§ 1251-1387). Soon after, the Love Canal toxic waste scandal brought into existence the Resource Conservation and Recovery Act (1976) (42 U.S.C. §§ 6901-6992k), the Comprehensive Environmental Response, Compensation and Liability Act (CERCLA), commonly known as Superfund (1980) and the Superfund Amendments and Reauthorization Act (SARA) in 1986 (Title 42, U.S.C., Chapter 103).
The Endangered Species Act (ESA) was prompted by a news account of the vanishing bald eagle (now a major species success story) and other prized species. Importantly, the ESA changed the focus of environmental legislation from preventing pollution and the preservation of wilderness to the legal protection of threatened and endangered species and the ecosystems that support them.
The ESA has had a dramatic initial impact on the management of federal lands, including National Parks and Wetlands, and thereafter the federal, state and local agency decision-making processes. In many cases, a combination of 1) the ESA and 2) a project sponsor’s failure to come up with appropriate habitat preservation and associated mitigation measures can prove the achilles heel of an otherwise worthy project.
§ 2.8 Military Base Reuse & Conversion
There is another, perhaps unlikely, source of environmental law: Upon the downsizing and consolidation of conventional Armed Forces in the United States in 1990, Congress passed the Federal Base Realignment and Conversion Act. BRAC provides a federal framework for the screening of bases for potential federal use and turn over to local governments.
California has been a defense state for many decades. As such, the state has experienced the closure of numerous military bases that have undergone the BRAC process. The Act has had significant influence on the environmental assessment and ultimate selection and approval of projects in California.
The economic goals of BRAC include the replacement of jobs in the community that will be lost with base closure and such other worthwhile objectives such as shelter for the homeless. The process is generally governed by Federal Property and Administrative Services Act of 1949, the 1990 Base Closure and Realignment Act, the McKinney Homeless Assistance Act and other Federal laws.
One of the most important elements of the reuse process is the environmental assessment and restoration of these former Army, Navy and Air Force bases for civilian use. Many of these bases had considerable contamination problems, such as the submerged lakes of pollution suspended between layers of clay (so called “perched lenses”) of gasoline, aviation fuels and cleaning solvents. The typical military base also contains high levels of rare metals and asbestos, stockpiles of weapons, as well as such things as randomly scattered and often shallowly buried, live ordinance from firing ranges and abandoned landfills. These can pose significant costs and risks for proposed infrastructure projects.
In many cases, local agencies are expected to assume considerable liabilities for the long term management of these bases upon turn over to civilian control. As such, the BRAC process requires considerable environmental knowledge, skills and planning for a successful rehabilitation of the federal property to civilian use.
It may also lead to considerable local controversies regarding the appropriate reuse of these bases. A leading example of this process is the Marine Air Station at El Toro, a magnificent 5.8 square mile parcel of land in the heart of Orange County, California. After years of controversy and several public votes, the County adopted a reuse plan featuring a vast urban park, along with housing and business park elements, rather than an international airport as initially proposed by local leaders.
§ 2.9 The California Environmental Quality Act (CEQA)
In California, the California Environmental Quality Act (CEQA) is the most important environmental statute affecting public agency decision-making. As stated by the California Resources Agency: CEQA applies to certain activities of state and local public agencies. A public agency must comply with CEQA when it undertakes an activity defined by CEQA as a “project.” A project is an activity undertaken by a public agency or a private activity which must receive some discretionary approval (meaning that the agency has the authority to deny the requested permit or approval) from a government agency which may cause either a direct physical change in the environment or a reasonably foreseeable indirect change in the environment.
Most proposals for physical development in California are subject to the provisions of CEQA, as are many governmental decisions which do not immediately result in physical development (such as adoption of a general or community plan). Every development project which requires a discretionary governmental approval will require at least some environmental review pursuant to CEQA, unless an exemption applies.
The environmental review required imposes both procedural and substantive requirements. At a minimum, an initial review of the project and its environmental effects must be conducted. Depending on the potential effects, a further, and more substantial, review may be conducted in the form of an environmental impact report (EIR). A project may not be approved as submitted if feasible alternatives or mitigation measures are able to substantially lessen the significant environmental effects of the project.
The CEQA Guidelines are found in the California Code of Regulations, in Chapter 3 of Title 14. They implement CEQA and incorporate and interpret statutes and judicial decisions. The CEQA process is internal to any agency considering a project and is subject to administrative and court challenge if not done in accordance with the statute, regulations and court decisions.
In addition to adopting the CEQA Guidelines and amendments thereto, the Secretary for Resources possesses the following responsibilities:
1) Makes findings that a class of projects given categorical exemptions will not have a significant effect on the environment;
2) Certifies state environmental regulatory programs which meet specified standards as being exempt from certain provisions of CEQA;
3) Receives and files notices of completion, determination, and exemption; and
4) Provides assistance in interpreting the provisions of CEQA and the CEQA Guidelines.
As a practical result, every project of any significant size must undergo a stringent evaluation and disclosure process known as an Environmental Impact Statement (EIS) where the Federal government is the lead agency or an Environmental Impact Report (EIR) where the State of California is the lead agency.
In each instance, a lead agency is appointed to lead the study effort and approve the final report and recommendations.
Initially, the agency will make a determination whether the project has so little impact that it is exempt from the process and no environmental impact statement is required (a so-called Negative Declaration.)
In certain instances, the EIR or EIS will be rejected by the lead agency or the elected public body overseeing the proposed project. In other instances, both the Negative Declaration and the substance and process of developing the environmental impact reports can be reviewed by the agency and eventually the courts through the process of administrative review. While courts are very interested in the process being upheld, as well as the fine tuning of reports to take into account full range of available mitigation factors, the legal system generally defers to the expertise of specialized agencies and the judgment of local elected officials regarding the substance of project decisions.
§ 2.10 Federal and State Environmental Assessment of Lands
It has been nearly fifty years since NEPA, CEQA and the ESA (and now BRAC) have been incorporated into the environmental evaluation processes for major infrastructure projects. One of the underrated benefits has been the widespread cataloging, inventorying, assessing and preserving of species and ecosystems, as part of comprehensive environmental impact statements for proposed infrastructure projects.
As a result, there is far more widespread public knowledge about the ecosystems and habitats of California and the nation that assist in broader project evaluation and understanding of the potential risks and rewards.
§ 2.11 Agencies with Permit Jurisdiction
CEQA was followed by State legislation regulating the impact of projects on public health, seismic safety, and air quality. Many of these statutes established agencies and committees that have a widespread influence on the evolution of a California infrastructure project. These agencies have permit authority over projects and, as a result, they exercise “Go - No Go” authority over many projects.
The process of attempting to overturn such an agency decision is extremely difficult, so the public owner must make every attempt to meet the spirit and letter of the law and regulations that will entitle it to the respective permits. It also does not hurt for the project to “sell itself” to these agencies as a net positive environmental project.
There are numerous major agencies in California with regulatory and approval authority for various aspects of major projects, including:
• California Environmental Protection Agency
• California Coastal Commission
• California Air Resources Board
• California Integrated Waste Management Board
• Department of Pesticide Regulation
• Department of Toxic Substances Control
• Office of Environmental Health Hazard Assessment
• State Water Resources Control Board
• Department of Fish and Game
• Cal Occupational & Health Administration
• The Division of the State Architect
In addition, there are numerous federal agencies with jurisdiction over California projects even where there are no available federal funds or grants, including:
• U.S. Environmental Protection Agency
• U.S. Department of Defense
• U.S. Army Corps of Engineers
• U.S. Coast Guard
Projects may be questioned and attacked by many consistencies. This is part of the natural and expected give and take of the political process. When the developers or sponsors of a project feel sufficiently threatened by citizens expressing their views in a public forum, they may elect to bring a libel or slander suit — a tactic, described above, as a Strategic Lawsuit Against Public Participation, or SLAPP Suit. Courts may dismiss or stay SLAPP suits under the provisions of California law known as the Anti-SLAPP Suit statute.
In addition to challenging the environmental impact reports, opponents of the project may also find that the project violates a provision of state or federal environmental law, including those statutes enumerated above regarding endangered species, air and water quality and associated regulated concerns.
Any good project planner will take these concerns seriously during project planning, engage the community in outreach and discussion of these issues, embrace environmental mitigation steps and meet and confer with opponents to negotiate or mediate environmental concerns, whether they are considered legitimate or not.
§ 2.12 Green Building & Sustainability
How do we design or construct a greener building? The U.S. Green Building Council (USGBC) has a list of guidelines for new construction that can help leaders design and construct buildings that are “carbon-neutral.” This designation of neutrality means that the building and its operational activities produce a net quantity of carbon emissions that equal zero. (Source of insert: Vendome Group — Construction Insider - 2008)
Suggestions from the USGBC — and touted by many professionals in the design and construction of today’s new buildings — include the following:
• Location: Use your location wisely. Buildings of the past did not take sufficient advantage of the path of the sun to use more natural lighting.
• Form: Minimize your environmental impact by using a design form that maximizes natural sunlight, stores heat, conserves space and uses materials that maximize energy usage.
• Windows: Capture all the daylight you can with many large windows, and make sure every part of your building takes full advantage of the daylight. Make those windows double-pane, energy-efficient windows, while you’re at it.
• Trees: The trees on your property can help offset the carbon dioxide you release by absorbing and storing some of it.
• Insulation: Proper insulation pays you back in dividends, as you get to keep what you heat.
• Go solar: Installing solar panels on a roof reduces the amount of electricity you need to purchase from fuel-burning power plants. Your roof could also take the form of a highly reflective “cool” roof, which can stay up to 70°F cooler than traditional materials during peak summer weather. The main benefit of a cool roof is the reduction in summertime air-conditioning expenditure. By minimizing energy use, cool roofs save money and reduce the demand for electric power and the resulting air pollution and greenhouse gas emissions.
• Light bulbs: Replace incandescent bulbs with compact fluorescents in your home and business. Fluorescent bulbs last 10 times as long and use considerably less energy.
• Natural heating, cooling and ventilation: The use of outdoor air flow into buildings to provide ventilation and space cooling is what designers mean by effective and natural cooling/heating system. Natural ventilation is a wholebuilding design concept. The aim is to control outdoor air supply while providing the required ventilation. Features of naturally ventilated buildings include exhaust vents located high in the building, with intake vents located low in the building, plus open building plans to facilitate air movement.
• Commissioning your building’s systems: Commissioning is the process of ensuring that systems are designed, installed, functionally tested, and capable of being operated and maintained to perform in conformity with the design’s stated intent. Since various systems throughout buildings are often installed by separate contractors, the commissioning process serves to verify the operability between these systems.
While the multifaceted strategies and methods of dealing with global warming are subject to debate, the practical reality is that construction professionals who are currently offering up greener building solutions that reduce costs, eliminate waste, and help boost a company’s image will be those getting the contracting jobs of the near future.
Web Sources: www.pewcenter.org; www.aia.org; and www.usgbc.org