- 1 Constraints in identifying river restoration project success
- 2 Benchmarking, Endpoints and project success
- 3 Benchmarking
- 4 Endpoints
- 5 Planning protocol for river restoration
- 6 Project Identification
- 7 Project Formulation
- 8 Setting benchmarks and endpoints
- 9 Setting project objectives
- 10 Cost benefit analysis
- 11 Risk assessment
- 12 Identification of projects
- 13 Project Implemenetation
- 14 Project monitoring and evaluation
- 15 References
Constraints in identifying river restoration project success
Little is known about the effectiveness of river restoration efforts despite the rapid increase in river restoration projects. Restoration outcomes are often not fully evaluated in terms of success or reasons for success or failure and this is, in part, due to weaknesses in the design and implementation stages of project planning for rehabilitation schemes. The review of concepts to measure the success of river restoration found that despite large economic investments in what has been called the “restoration economy”, many practitioners do not follow a systematic approach for planning restoration projects. As a result, many restoration efforts fail or fall short of their objectives, if objectives have been explicitly formulated. Some of the most common problems or reasons for failure include:
- Not addressing the root cause of habitat degradation
- Poor or improper project design, skipping key design steps
- Expectations not clearly defined with measurable objectives, therefore project success is difficult to evaluate through monitoring (Bernhardt et al. 2007)
- Not establishing reference condition benchmarks and success evaluation endpoints against which to measure success
- Failure to get adequate support from public and private organizations
- No or an inconsistent approach for sequencing or prioritizing projects (Roni et al. 2013)
- Inappropriate use of common restoration techniques because of lack of pre-planning (one size fits all) (Montgomery & Buffingtion 1997)
- Inadequate monitoring or appraisal of restoration projects to determine project effectiveness (Roni & Beechie 2013)
- Improper evaluation of project outcomes (real cost benefit analysis)
Furthermore, a review of 671 European case studies collated for REFORM WP1 Deliverable 1.3 (Wolter et al. 2013) identified only a small number of case studies reported ecological success or failure: many were either unclear in their findings, the restoration works were not monitored or no information was given in the reports (Figure 1). The reviewed identified that only 52% had been monitored and from this only 3% recorded physio-chemical success, 8% recorded morphological success and 17% recorded biological success. This remarkably low adherence to what would seem good project management practice is possibly attributable the limited guidance for evaluating the success of restoration projects.
Benchmarking, Endpoints and project success
Overall, evaluating how successful restoration measures have been, as well as determining reasons for success or failure are essential if restoration measures are to meet obligations under the Water Framework Directive. Setting benchmarks and end points that are linked to clearly defined project goals is considered the most appropriate approach to help measure of success (Buijse et al. 2005).
Benchmarking as a tool should be feasible, practical and measureable to help guide future decision support tools. One of the first steps is to establish benchmark conditions against which to target restoration measures. Benchmarking uses representative sites otherwise known as ‘reference sites’ on a river that have the required ecological status and are relatively undisturbed; this is then used as a target for restoring other degraded sections of river within the same river or catchment. This requires:
- Assessment of catchment status and identifying restoration needs before selecting appropriate restoration actions to address those needs
- Identifying a prioritization strategy and prioritizing actions
- Developing a monitoring and evaluation programme
- Participation and fully consultation of stakeholders
It is imperative that endpoints accompany benchmarking in the planning process to guarantee the prospect of measuring success because endpoints are feasible targets for river rehabilitation. It is important to note that endpoints are different to benchmarks, this is because other demands on the river systems also have to be met and references can only function as a source of inspiration on which the development towards the endpoints is based (Buijse et al. 2005). Given that benchmark standards cannot always be achieved, especially on urban rivers, endpoints will therefore assist in moving restoration effort forward through application of the SMART approach, to decide what is achievable and what is feasible. There is a need to distinguish endpoints for:
- Individual measures
- Combination of measures
- Catchment water bodies
- River basin districts
Planning protocol for river restoration
With an increasing emphasis on river restoration comes a need for innovative tools and guidance to move decisions based largely on subjective judgments to those supported by scientific evidence (Boon & Raven 2012). The restoration planning protocol developed uses project management techniques to solve problems and produce a strategy for the execution of appropriate projects to meet specific environmental and social objectives (Figure 2). The procedure is process driven in its development and makes use of various project planning tools (e.g. PDCA, DPSIR, conflict resolution, environmental impact assessment and logical framework, SMART and participation ladders) to:
- Diagnose problems and produce a strategy for their remediation;
- Provide knowledge of the technical policy and background to conflicts of multiple use of resources;
- Develop a plan based on benchmarking and endpoints for setting specific and measurable targets with objectives as defined by the institutional, regional, national policy;
- Ultimately develop action plan with actions and targets, whilst recognising the need for an integrated approach to management of resources to minimise conflicts and optimise use.
Figure 2. Proposed planning protocol for restoration projects - yellow coloured boxes represent steps in the DPSIR approach to management intervention.
Projects should typically proceed through three main phases associated with the project cycle (e.g. Skidmore et al. 2011):
- Project identification: which establishes the purpose and need for restoration, puts the project in a watershed context, and articulates the specific intentions of a project
- Project formulation: which describes the details of the project and how it will be implemented and the project objectives accomplished
- Project Implementation & monitoring: which includes the actions taken to complete the project, checking to see that the project was implemented as designed, and evaluating whether the project had the desired habitat and biological effects.
Table 1 (File:WP5 1Table 1.pdf) contains an overview of steps on the planning protocol outlined in Figure 2.
Project identification is the stage at which the initial restoration project proposal is conceived and formulated. The identification phase may be divided into two fundamental aspects. In the first the restoration project concept is considered in relation to:
- The overall status of the aquatic ecosystem functioning and the ecological status or potential;
- The regional or national policy and WFD priorities (see this link).
The first step provides an understanding of the current status of the ecosystem functioning and ecosystem services in the management zone to establish the baseline against which to develop any restoration project (equivalent to the DPSIR State assessment). The basic information required includes, but is not exclusive to:
- Background geography and landscape topography, political domains, climate and general infrastructural development;
- Habitat modification and geomorphological alteration;
- Hydrology, including modifications to flow regulation, abstraction and other water uses;
- Flood defence;
- Fisheries, recreation and conservation;
- Water quality;
- Land use/navigation and mineral extraction;
- Urban, agricultural and industrial development.
In the second aspect of the identification phase the relevant policy issues are considered, notably:
- The overall justification for the project (perspectives, development objectives);
- The likely target groups and impact beneficiaries, as well as those who might be adversely affected;
- The key factors influencing the likely success and failure of the project.
Restoration objectives should be clearly defined adopting a river basin-wide approach, and have been developed from high priority WFD and national policy objectives (equivalent to the DPSIR Drivers assessment). This intrinsically moves the existing approach from being issue-driven towards an emphasis on forward planning. Typical WFD policy objectives include:
- Achievement of GES or GEP;
- Conservation & efficient exploitation of resources;
- Contribution to species conservation objectives;
- Creation of regional employment and maximising social benefits;
- Regional development (regional and multi-lateral cooperation);
- Establishment of the legal and administrative framework for regulation;
- Assessment of environmental, economic and social impacts;
- Maximising ecosystem health.
The third aspect concentrates on the techniques used to measure the viability of the restoration project as it evolves through the phases of the project approach. One of the most commonly used techniques to structure the process described above is the logical framework approach (Anon 1982; Table 2). The technique is useful in setting out the design of the restoration project in a clear and logical way so that any weaknesses that exist can be brought to the attention of the planners. Starting with the aim of the project, a series of objectives, outputs and inputs are developed down the first column at the left-hand side of the page. The second column addresses the indicators (endpoints) that have been determined at the outset of the project and how they can be verified as the project is developed further through the various phases of the project approach. The final column assesses the risks and assumptions which underpin the elements described in the first two columns. As the restoration project develops the logical framework will be modified to take account of new information likely to affect the project elements. Any deficiencies may then be remedied at an early stage, or if insuperable, the restoration project may be discounted. The logical framework technique emphasises the value of choosing measurable indicators or endpoints which can be assessed throughout the life of the project, and also instructs the planners to assess carefully the risks and assumptions upon which the project is based.
Estimates, and qualitative indicators laid down in the identification phase will be refined and examined in detail. The technical, financial, economic, institutional, social and environmental criteria should all be revised, especially in the light of the requirements of detailed budgeting and monitoring which hitherto will not have been investigated. In the project identification phase the project planners should be concerned with the suitability and feasibility of the project, in the formulation phase the emphasis shifts to the acceptability of the project and the desired outcomes. The logical framework will be expanded and upgraded to describe not only the objectives, principal issues, and definition of project components, but also the detailed costing’s, proposals for organisation and management arrangements, and economic and financial assessments. At this stage base-line survey work to substantiate the technical components of the project should be undertaken.
The formulation of restoration projects naturally falls out of comparison of the status of the aquatic ecosystem and the overall regional and national policy objectives. With respect to meeting WFD objects of GES and GEP, the current ecological status of the aquatic system as whole is compared against the environmental aspirations and targets. This will highlight the contrast between current status and functioning of the ecosystem and aspirations for the water body, and thus draw out the aspects of the water body that will need to be maintained, improved and developed, and identify the issues and constraints to achieving the target. Typical areas to review include:
- Natural resources as strategic assets;
- Focus on the natural resources of current and/or potential economic importance;
- Sustainable development;
- Extent to which effective natural resources management is currently being achieved and examine future proposal in the light of technical and institutional capabilities;
- Present broad management options as the basis for restoration strategies;
- Identify externalities that impact on the ecological status and responses to these externalities;
- Identify economic and functional linkages between restoration actions and between other sectors (e.g. hydropower and navigation);
- Identify potential for combining inputs to optimise resources or increase benefits;
- Conflicts between interest groups;
- Identify actual and perceived conflicts and ways to alleviating them;
- Assess effectiveness of authorities and make proposals for improvement;
- Legislative framework;
- Assess how legal framework adds or detracts from successful economic performance;
- Monitoring and enforcement of management regulations;
- Review social issues in relation to restoration objectives;
- Human resource capabilities;
- Capacity of research institutions and institutional framework for dissemination of information;
- Identify existing skills and aptitudes can be developed to generate increased benefits;
- Examine record of introducing new technology and management ideas.
At this stage a problem tree analysis is undertaken to review the cause effect of key issues (Figure 3).
Figure 3: Structure of problem analysis tree
This identifies substantial and direct causes and effects of the focal problems. It is critical that existing issues are identified as well as future issues that will arise because of knowledge of planned development, e.g. a new hydropower dam being constructed. A problem/issue is not the absence of a solution (e.g. no land available for reinstating the natural water course) but an existing negative state (e.g. obstructions to fish migration). Conflicts between user groups will potentially be highlighted. Throughout the analysis there is a need for comprehensive local consultation with between stakeholders to understand their needs, motives and drivers. An example of a problem analysis is shown in Figure 4 with reference to channelization and disconnection of the floodplain.
Figure 4: Example of problem analysis related to channelization and disconnection of the floodplain.
It is critical at this stage to set SMART objectives for the proposed restoration action. This should encompass establishing target conditions based on an understanding of what is technically feasible, socially acceptable and economically viable. The procedures of setting benchmarks and endpoints are critical to defining the outcomes of this stage, i.e. setting project objectives.
Setting benchmarks and endpoints
Establishing benchmark conditions against which to target restoration measures is a fundamental step in all restoration project planning but it is seldom adopted. Setting benchmarks draws on the assessment of catchment status and identifying restoration needs before selecting appropriate restoration actions to address those needs, identifying a prioritization strategy and prioritizing actions (see REFORM WP6), and developing a monitoring and evaluation programme.
Goals and objectives need to be set at multiple stages of the restoration process, and there are multiple steps within each stage, but the initial step is to identify endpoints and benchmarks against which to measure performance. This needs be reviewed against reference conditions, to determine appropriate targets for restoration, rehabilitation and mitigation activities.
The process of benchmarking can be broke down into a number of steps:
- Reference condition: Deriving reference criteria – need to establish reference conditions of specific river types or river styles as defined by river characterisation. This may not be the pristine state but should describe the state or value of a defined ecological attribute if the system had not been disturbed by the specific pressure of pressures. It may well be defined by nearby undisturbed (by said pressure[s]) reaches of rivers that is achieving GES or GEP, i.e. an ecosystem with ecological integrity commensurate with what meeting societal aspirations.
- Expectation: Transfer reference conditions to end points for target systems – different for each river style including temporal and spatial dimensions. This will require comparison of status against objectives for restoration that are appropriate to accommodate variability in river style/types (river characterisation). Establishing endpoints identifies characteristics of concern that reflect the overall restoration goal.
- Baseline condition: Undertake deficit analysis (to identify what hydromorphological limitations and processes are constraining the recovery of the biota) and explore the potential for restoration to establish ‘endpoint’ target conditions.
- Once the end points have been established these restoration targets need integration into wider catchment-based activities to deliver win-win scenarios (e.g. flood mitigation, hydropower, agriculture, navigation) and take due account of the cost and benefits, specifically in relation to ecosystem services delivery, to ascertain the most effective measures to meet specific objective.
Setting project objectives
From the review of issues and deficit analysis, it should be possible to identify development options and future restoration projects. This is most easily achieved using an objectives tree analysis (Figure 5). Here the problem tree (Figure 4) is transformed into a tree of objectives (options to resolve the problem). For example, for the problem “If cause A, then effect B” the option would be “Means X to achieve end Y”. The aim is to reformulate all elements in the problem tree into positive desirable conditions. This process is designed to help the project manager think about the key aspects of the river restoration project and what the project is setting out to achieve, and to recognise the inherent complexity. Key questions to consider include:
- Is the main aim of the project to improve the physical processes of the river or to increase biological diversity in defined areas?
- If the focus is to increase river forms and processes, what will be the benefit for the ecology (specific fauna and flora and, where appropriate, part(s) of life cycle(s))?
- If the focus is to increase ecological (habitat) diversity for a range of fauna and/or flora, which parts of the life cycle are being aimed to restore for and what physical river features are expected to develop to support this goal?
- Are the objectives SMART:
- Clear (Specific)? - Quantifiable (Measurable)? - Achievable, Realistic and Time-bound?
- Have quantitative or qualitative indicators been established that provide a simple and reliable means to measure achievement, reveal the changes connected to an intervention, or help assess the performance of an organization against the stated target. Such performance indicators are used to assess and measure the progress related to an expected result or an aspect of it and to identify to what extent beneficiaries/target groups have been reached and such be defined in the logical project framework (Table 2).
Figure 5. Example of objectives tree analysis related to channelization and disconnection of the floodplain.
The outputs can then be used to construct a matrix table that defines the issue (cause and effect) and reviews the potential options for resolving the issue, the best restoration measure to achieve the desired goal (Table 3). Since there is likely to be more than one option (measure) or a combination of measures to resolve an issue, the advantages and disadvantages of each should be considered and their inter-linkages explored. In addition this analysis should include the feasibility of achieving the outcome of the stated option both from a technical as well as a financial perspective, but also to identify win-win scenarios.
Table 3. Structure of table to undertake options analysis
It should also be noted that many factors will influence the decision on design options for river restoration schemes (Table 4). Bear in mind that any designs are site-specific and depend on local circumstances.
Table 4. Important considerations prior to, and during construction
Cost benefit analysis
Rivers, however, are of high socio-economic and socio-cultural importance and provide “a myriad of benefits to society” (ecosystem services Vermaat et al. 2013). Unfortunately the benefits generated by rivers are difficult to quantify and evaluate but this is fundamental to undertake cost benefit analysis of the most appropriate measures for achieving the best outcomes in terms of WFD delivery. Numerous methodologies have been developed to undertake cost benefit analyses that are applicable to help decide on the most appropriate measures to meet a desired outcome. It is not proposed to discuss these here but refer the reader to REFORM deliverables associated with WP 1 and 5 (D1.4, D5.2) and suffice to say that part of the planning procedure should include cost benefit analysis of the proposed actions to maximise of the benefits accrued.
Before any proposal for a restoration scheme is approved, a thorough assessment of the risks associated with the exercise must be undertaken. Two aspects of risk should be considered, 1) Risks associated with failure implement the project and 2) Risks associated with project not achieving the expected ecological outcomes or endpoints. This aspect will not be dealt with in detail because it is the subject of a separate report (REFORM D5.4).
Identification of projects
Once the management plan has been formulated, and adequate consultation has been made with Government departments, institutions, user-groups, industry and the public, it should be possible to draw up action plans for the future development of the restoration meant for improving ecological status or delivery of improved services. This includes justification and prioritising of projects, project formulation, and outlining costs and budgetary considerations.
The implementation phase is characterised by the detailing of work plans and financial arrangements. The logical framework drafts, which will have been refined several times since identification, will be translated into activity schedules. Disbursement of project funds into budget heads will be implemented and all the monitoring and control mechanisms should be in place.
Key to successful implementation includes:
- Expanding the manager’s view of who is affected by aquatic resource management (stakeholder);
- Identifying and understanding stakeholder views;
- Seeking compromise between competing and conflicting demands;
- Improving communication between managers and stakeholders.
Restoration plans should not be based just on technical issues and their effectiveness or limitations. They must involve:
- Regional policy framework;
- Societal and prevailing ideas and values;
- Institutional frameworks, i.e. fit within the regulations and legislation.
Project monitoring and evaluation
Monitoring and evaluation plays a key role within the framework because it enables identification of river restoration project success. Pre-monitoring helps identifying restoration goals, while restorations goals help defining specific monitoring objectives to guide the development of a monitoring and evaluation programme. Monitoring elements (usually WFD Biological Quality or HYMOElements) should be chosen with a rational to focus on those that respond to the restoration action and address the question outlined in the hypothesis (REFORM Deliverable 3.1). Selecting a monitoring design is essential to make the data meaningful for evaluation, i.e. determining the spatial and temporal scale for monitoring and identification of treatment, control and reference sites. Monitoring not only helps to define benchmarks and endpoints at the start of a project but also determines when the endpoint of a project has been reached. However, it can be difficult deciding when the restoration process is ‘complete’ and therefore, it is essential that an impact assessment monitoring design is employed to provide evidence, in statistical terms that an endpoint has been reached. A variety of impact assessments techniques are available to detect environmental change for rehabilitation project whose data collection methods differ spatially and temporally. A replicated BACI design is the most powerful design because it includes replication in both space and time and this is recommended. A resource calculation can be applied to determine how many years pre and post monitoring is required to isolate the environmental impact from natural variability.
The evaluation phase, for a rehabilitation project which has undergone the initial stages of the project approach, assesses the overall project effects (intentional and unintentional) and the sectoral impact of the project. Evaluation is only possible where a series of measurable indicators or endpoints has been established for the project, hence the value of establishing and updating the logical framework throughout. The evaluation phase will use the indicators to gauge how far the restoration project has developed in relation to the initial objectives and defined endpoints. Again the analysis will come back to the logical framework which was established at the outset and has been subsequently refined through the preparation and appraisal phase. It is the indicators laid down in the logical framework which are used to monitor the restoration project during implementation. In addition, the implementation criteria will be used in ex-post evaluation which takes place some years after completion of the restoration project is complete. Assuming all goes well and the project is implemented, the evaluation phase should provide a steady feedback of information and results which will be useful in other restoration project situations. Progress reports should be formally produced and assessed, focussing on the key indicators of the project in order that lessons may be learned and problems avoided in future restoration programmes.
Anon, 1982. The logical framework approach (LFA). Norwegian Agency for Development Corporation, Oslo, Norway.
Bernhardt, E.S., Palmer, M.A., Allan, J.D., Alexander, G., Barnas K., Brooks, S., Carr, J., Clayton, S., Dahm, C., Follstad-Shah, J., Galat, D., Gloss, S., Goodwin, P., Hart, D., Hassett, B., Jenkinson ,R., Katz, S., Kondolf, G.M., Lake, P.S., Lave, R., Meyer, J.L., O’Donnell, T.K., Pagano, L., Powell, B., & Sudduth, E., 2005. Synthesizing US river restoration efforts. Science 308 (5722), 636-637.
Boon, P. J., & Raven, P. J., 2012. Front Matter, in River Conservation and Management, John Wiley & Sons, Ltd, Chichester, UK. doi: 10.1002/9781119961819.fmatter.
Buijse T., Klijn F., Leuven R.S.E.W., Middelkoop H., Schiemer F., Thorp J.H. & Wolfert H.P., 2005. Rehabilitation of large rivers: references, achievements. Archiv für Hydrobiologie Supplement Large Rivers, 155, 715–720.
Montgomery, D.R., and Buffington, J.M., 1997. Channel-reach morphology in mountain drainage basins, Geological Society of America Bulletin 109, 596-611.
Roni, P. & Beechie, T., 2013. Stream and watershed restoration – A guide to restoring riverine processes and habitats. John Wiley & Sons, Ltd.
Roni, P., Liermann, M. & Steel, A., 2003. Monitoring and evaluating responses of salmonids and other fishes to in-stream restoration. Pages 318–339 in D. R. Montgomery, S. Bolton, and D. B. Booth, editors. Restoration of Puget Sound rivers, University of Washington Press. Seattle.
Skidmore, P.B., Thorne, C.R., Cluer, B.L., Pess, g.R., Castro, J.M., Beechie, T, J., & Shea, C.C., 2011. Science base and tools for evaluating stream engineering, management and restoration proposals. NOAA Technical Memorandum NMFS-NWFSC-112.
Vermaat, J., Ansinkb, E. Wagtendonkb, A. & Brouwerb R. 2013. Valuing the ecosystem services provided by European river corridors – an analytical framework. Reform Deliverable D 2.3.
Wolter, C., Lorenz, S., Scheunig, S., LehmannN., Schomaker L. & Nastase A. 2013. Review on ecological response to hydromorphological degradation and restoration, Reform Deliverable D 1.3.