Evaluation in Planning: An Investigation into Plan Quality and its Application to Official Plans in the Ontario-Greater Golden Horseshoe (GGH) Region

Abstract

Evaluation is an established activity in public sector organizations. Evaluation is rooted in the principle that government interventions should have demonstrable benefits. In the realm of planning, there are two broad areas of evaluation: (1) planning evaluation and (2) plan evaluation. Planning evaluation is concerned about evaluating planning processes and planning practice, while plan evaluation focuses on assessing plans and their outcomes. Plan evaluation includes assessing the quality of plans, the success of plan implementation, and the outcomes of plans. While both forms of evaluation remain relatively unexplored, when compared to other areas of planning literature, the gap in knowledge regarding plan evaluation is particularly pronounced. This research contributes to our understanding of plan evaluation, particularly in regards to assessing the quality of official plans.

This dissertation follows an article-based format and includes four manuscripts. At the time of submission, two manuscripts (Manuscripts 1 and 2) were published in the Journal of Planning Practice and Research and Journal of Planning Education and Research, while the remaining two manuscripts (Manuscripts 3 and 4) were under review at the Journal of the American Planning Association and Journal of Planning Education and Research. Manuscripts 1 and 2 were co-authored with Dr. Mark Seasons, Advisor.

Manuscripts 1 and 2 reviewed the existing literature on evaluation in planning. Manuscript 1 examined the factors that contribute to the under-use of plan outcome evaluation in local government planning practice. The concept of evaluation was explained and the relationship that exists between program evaluation and plan evaluation was explored. Manuscript 2 reviewed the major approaches of program evaluation and evaluation in planning, including formative, summative, ex ante, on going, and ex post evaluations. The challenges to evaluating plans were also discussed including the reliance on ex ante evaluations; a lack of outcome evaluation methodologies; the attribution gap; and institutional hurdles. Areas requiring further research were highlighted in both of these manuscripts.

Manuscripts 3 and 4 discussed the results from survey research and content analysis. Manuscript 3 discussed the results of a web-based anonymous survey administered to 290 municipalities across the province of Ontario. The findings indicated that practicing planners generally regard plan quality as important; researchers and practitioners should not treat plan quality principles equally; and implementation and monitoring and evaluation principles were undervalued as being very important contributors to plan quality when compared to other principles.

Manuscript 4 discussed the findings from content evaluations of 63 official plans found in the Greater Golden Horseshoe (GGH) region using 70 indicators to measure nine plan quality principles discussed in the literature. The findings indicated that goals and policies were the strongest principles; fact base, monitoring and evaluation, and public participation were the weakest principles; implementation and inter-organizational coordination were somewhat weak; and plan organization and presentation and legislative requirements were reasonably strong.

Based on the findings from the literature review, survey research, and content analysis, several strategies to enhance the quality of official plans are proposed, including: strengthening the importance of the provincial government’s role in planning as a means of improving the quality of local official plans; enhancing implementation and monitoring and evaluation provisions in planning initiatives; better describing the empirical foundation and participation process used to inform plan development; and extending plan quality evaluations to planning practice.