Understanding the Value Proposition of City Networks: Towards a Framework for Measuring Impact
On 14 July 2021 city network practitioners, city officials from international relations departments, experts, and scholars gathered in an online webinar organized by CIDOB with the support of the Barcelona City Council and in collaboration with Metropolis and the Chicago Council on Global Affairs. The aim was to discuss the value of the services provided by city networks and to examine potential methods for measuring their impact on improving the urban policies of their members. Building on the results of the webinar, this briefing examines some of the limitations networks face in fulfilling some of their essential functions (i.e., advocacy, knowledge dissemination, policy implementation) to their full potential. It also formulates some preliminary guiding principles for elaborating a framework to evaluate the impact of the services provided by city networks on its members.
1. Introduction
City networks have become a permanent fixture of the landscape of global urban governance. They have proliferated to an extent hardly imagined some decades ago, and today they constitute a complex ecosystem composed of more than 200 entities and counting (Acuto and Leffel, 2021). In essence, city networks can be defined as institutionalized structures that facilitate cooperation among local (and/or regional) governments, which can join (or leave) the network voluntarily. Moreover, these networking spaces provide various services to their members and play a decisive role in helping them to address shared challenges collectively.
Over the last decade, the range of activities and services on offer has been significantly expanded and professionalized, while also acquiring new levels of sophistication. This has led to the formation of complex organizations that operate through large and highly professionalized teams, where the line between technical and political leadership is blurred. Initially, associations of local and regional governments were spaces mainly devoted to the exchange of information, cooperation, and lobbying activities of their members worldwide. Nowadays, they have been turned into real laboratories for innovation, policy diffusion and implementation. They are even seen as critical engines allowing the ‘glocalization’ of urban governance to occur.1
Studies seeking to understand how these organizations deliver and perform these services have proliferated in recent years (for instance, Oomen, 2020; Fernandez de Losada and Abdullah, 2019, Papin, 2019; Rahidi and Patt, 2018). Yet, most practitioners would agree that there is still poor understanding of how the services provided by city networks are perceived, experienced, and valued by their members,2 or whether they are adequately responding to their expectations and requests. Neither is there anyclear framework for measuring their impact and effectiveness in transforming local realities on the ground. For instance, to what extent does joining a particular city network improve the formulation of local climate, gender, or digital policies? Are cities making the most of the services on offer? And if not, what are the main obstacles and limitations they are facing? How can the value proposition of city networks be enhanced to serve their members’ needs better?
Those are not trivial questions, since being able to evaluate impact could provide a clearer picture for cities on the added value of joining or the convenience of withdrawing from one network or another. This is relevant if we consider both the current overcrowded ecosystem of city networks and the over representation of certain cities, some of which belong to 40-50 networks. Furthermore, measuring the return on efforts invested in these transnational networking spaces could also prove to be a valuable instrument for legitimizing the internationalization policies and strategies of local governments in the eyes of their citizens. Finally, this is a relevant issue not only for cities but also for the increasingly professionalized network secretariats, which aim to provide maximum value for members and, in order to serve them better, are also open to reformulating their functions and services.
This Briefing seeks to shed light on these issues by discussing the value of the services city networks provide and examining potential methods for measuring their impact on improving the urban policies of their members. The analysis builds on the results of the online webinar “Understanding the Value Proposition of City Networks and Measuring Impact” organized by CIDOB on 15 July 2021 with the support of the Barcelona City Council and in collaboration with Metropolis and the Chicago Council on Global Affairs. The event brought together city network officials and practitioners, city officials from international relations departments, experts, and scholars to share their insights on the subject.
The paper is structured in to three sections. The first provides an overview of the main functions currently performed by city networks. The second examines some of the obstacles networks face in fulfilling these functions to their full potential, and draws attention to the main challenges identified by both city networks and their members. The third makes suggestions on how to formulate an adequate framework for measuring the impact of the services offered by city networks. The aim of this briefing is not to provide conclusive findings on this subject, but rather to launch a discussion that will be continued in the coming months by CIDOB in collaboration with Metropolis.
2. The value proposition of city networks: advocacy, knowledge dissemination, policy implementation
Broadly speaking, most city networks perform three main types of functions: advocacy, knowledge dissemination, and policy implementation. First, with advocacy, they represent the collective interests of their members vis-à-vis other tiers of government (i.e., national, regional, global) and lobby on their behalf to include the urban voice in global policymaking. Advocacy efforts are often directed towards strengthening and institutionalizing the participation of local and regional governments in international frameworks (i.e., the often-quoted discussion over “getting a seat at the global table”).
Yet, city networks are also instrumental in raising awareness on particular issues in intergovernmental forums and negotiation processes. Examples, here, are the campaign for the SDG 11 (the so-called “urban SDG”), efforts to include the right to the city in the New Urban Agenda, and actions aimed at boosting the ambition of the global decarbonization goals (e.g., the Cities Race to Zero campaign). This entails a mediating role between the global and local levels, where city networks are instrumental in translating global frameworks into local realities and vice versa.
Second, knowledge dissemination is often seen as the “bread and butter” of city networking, since information is arguably one of the most valuable assets these organizations possess. Indeed, most initiatives are to a greater or lesser extent directed towards stimulating mutual cooperation and exchange among member cities and their peers, in a process where city networks act as platforms for knowledge sharing and policy learning.
This is facilitated, among other things, through the articulation of networking spaces that enable interaction and exchange of insights among members (i.e., discussion forums, conferences, workshops, seminars), exchanges with solution providers (i.e., private sector, knowledge-based institutions, CSOs, etc.), and other myriad initiatives such as fostering peer-to peer exchange programmes, compiling best practices, publishing databases and thematic reports, promoting learning initiatives, etc. The underlying rationale here is to learn about what has (and has not) worked in other places so as to avoid reinventing the wheel for comparable challenges.
Third, policy implementation Refers to the fact that beyond facilitating peer-to-peer learning and the exchange of information, most city networks devote significant efforts to assist in building the capacity of their members to respond to a variety of urban challenges or to implement particular policies. In this regard, city networks often act as implementing agencies for regional or global policy frameworks and stimulate action to reach these goals. Some outstanding examples are the localization of the 2030 Agenda for Sustainable Development, the implementation of global climate targets, promotion of the right to housing, and driving rights-based digitalization processes.
Support from city networks with policy implementation requires resources and skills from multiple actors that go far beyond the provision of a platform for peer-knowledge exchange, and that include the use of technologies (e.g., software to measure GHG emissions), expertise, contacts, know-how, monitoring standards, and even financial resources. Other typical activities performed by these organizations include drafting guidelines and roadmaps, and establishing benchmarking and certification systems to monitor and recognize progress. It is also an increasingly common practice to provide consulting services and direct technical support to strengthen members’ capacities, to assist in brokering partnerships with the private sector and/or civil society organizations, and to facilitate access to funding in order to implement specific innovative and replicable projects.
3. Making the most of what is on offer
As previously argued, the services portfolios of city networks are steadily diversifying into a broader range of products and activities. This is partially explained by the need to bring value and deliver meaningful benefits to their members. Yet, the flip side of this trend is that member cities are subject to increasing pressure to partake in a frequently excessive agenda of meetings and programmes. To get an idea of the dynamism of the ecosystem, one only has to look at the events calendar of any of these associations. There is virtually no week without a packed agenda of webinars, conferences, or learning sessions. Since many cities are members of several networks, the level of engagement required becomes simply untenable for most city authorities. Worse still, city networks' efforts in delivering these services are not always matched by tangible results with measurable impacts.
Hence, one of the first obstacles is the limited capacity of cities to respond to the requirements of networks.3 The case of the organizational evolution of C40 is illustrative. In only 10 years, the C40 staff has multiplied by six (to 250 professionals at present), following expansion of the programmes and focus areas of the organization. Since it is a practical and evidence-oriented network, the staff is tasked with collecting information from member cities to produce research reports or monitor progress in their climate commitments. The problem is that the number of member cities (always around 90) has not grown accordingly, and this frequently leads to a situation in which city officials feel unduly pressured to deliver and meet the mandatory participation standards set forth by C40. Again, it is essential to recall that most of these member cities belong to more than one city network, all requesting some sort of action or information from their members (i.e., fill in a survey, draft a report, organize a site visit, etc.). It has therefore become imperative for this organization to find a way to avoid putting too much pressure on cities as a necessary condition towards remaining relevant for them.
Addressing this issue requires a comprehensive approach that takes into consideration the broader ecosystem of city networks in all of its complexity. An often-voiced concern among the community of practitioners is how collaboration among (usually competing) local governments associations could be improved in order to enhance the efficiency of the services they deliver and the impact of the initiatives they champion. This is a thorny issue, for it is an open secret that coexistence in the overcrowded ecosystem of city networks is not always easy. Indeed, the political space where city networks cohabit is structured in such a way that they are often forced to vie for resources, visibility, impact, and members (Fernandez de Losada andAbdullah, 2019). Accordingly, the still imperfect cooperation among city networks results in major inefficiencies that hamper their ability to deliver on their goals. To mention just one recent example, in July 2021 the main regional European city networks simultaneously launched different position statements on the occasion of the new EU Green Deal and climate ambitions, which had a poor impact on their intended audience. Many acknowledged that a unified statement would have been more effective. As for improving coordination among the main global city networks, some hopes are pinned on the Global Taskforce of Local and Regional Governments.
Without improved coordination, it is unlikely that city networking will move beyond international trips, public relations, and exchanges of ideas that may be inspiring but go nowhere, as they are never fully translated into action. Furthermore, meaningful city diplomacy requires building the internal capacity of municipal teams to participate in these spaces (i.e., acquiring the relevant skills, finding the right person or team, etc.) and overcoming interdepartmental coordination issues. In fact, in most of the cities there is no single person tasked with oversight of the city’s participation in different networks, which leads to inefficiencies or missed opportunities. The entry points for networks are usually international relations departments or the mayor’s office. However, in many cases due to lack of capacity or political interest, the benefits of being part of a network are not fully transferred to the different municipal departments.
One example of good practice can be learned from the City of Johannesburg,4 which has embarked on a project to measure all global commitments the city has taken on. Measuring implies having a clear picture of what the objectives of each commitment are, and pursuing quantifiable, realistic targets. Approaching participation in city networks from this perspective may lead to better results on the ground.
Last but not least, given the constant reality of increasing pressure on resources, determining the impact of the actions and activities performed by any organization is imperative. This requires measuring and quantifying, as it makes for more effective advocacy at the global level and may also accelerate the implementation and monitoring of specific policies. In this sense, one cannot overlook the scattered efforts by city networks themselves to appraise whether their activities are valuable to their members, either in the form of surveys or through contracting external independent assessment reports. For example, the network Metropolis, which has been basing evaluation of its impact in terms of expenditures, activities, and mobilization of people, has introduced, for the first time, a point on transparency and accountability in its Strategic Action Plan 2021-2023 which may lead to a new way of reporting impact.
Yet, measuring performance is no easy task, and we still lack an appropriate framework to assess the benefits (and downsides) of city networking and measure the impact of its initiatives and services on members. The remaining section of this briefing makes some suggestions for developing such a framework.
4. Towards an adequate framework for measuring impact
a. Determining realistic prospects
Any evaluation requires determining realistic prospects about what can be successfully measured and what can be reasonably expected from participation in a city network. In this regard, not all the functions identified in the first section of this briefing (i.e., advocacy, knowledge dissemination, policy implementation) may be equally easy to evaluate. For instance, gauging the impact of a particular campaign at the global level is significantly more complex than measuring the results of capacity-building programmes. Moreover, outcomes and impacts of many of the initiatives promoted by city networks in the short term may be quantifiable but irrelevant (e.g., metrics of web visits, retweets, number of activities or policies adopted) or even detrimental to assessment of unquantifiable questions of effectiveness that can only be perceived in the long term (e.g., improvement of the image and global positioning of a particular city or what has been achieved by influencing decision-makers).
Monitoring and evaluating thus requires taking into consideration both short-term and long-term change, and also assuming that many effects will be underestimated because of the difficulty in measuring them. This is particularly true for long-term evaluation, which may require costly data collection strategies. Hence, short and medium-term goals will frequently have to be used as a surrogate for understanding the more important, but difficult-to-measure, impact of a particular activity or campaign.
This is not to say that efforts to design effective evaluation frameworks and impact reports should not be encouraged. In fact, in these cases the perfect can easily be the enemy of the good. There are already some good practices regularly performed by city networks that could be seen as a first step in the right direction. A case in point is the reporting system of C40, which uses scoring cards to place some pressure on its member cities to deliver on their climate commitments. The organization then annually recognizes the three most active cities in each region.
Similarly, Metropolis’ annual report includes a section called “Engagement of Our Members” that details mobilization and performance of the different members of the association. By the same token, Energy Cities publishes an annual impact report that “is not about what we have done, but about the impact we have had on the people, places and policies that determine how we live”. Arguably, these initiatives provide only a limited picture of the level of involvement or participation of cities in these networks and tell us little about the added value of these organizations to its members. However, they can also be seen, perhaps, as a first step towards adopting more sophisticated evaluation frameworks in the future.
b. Pointing to the right goals and targets
Before gathering and measuring data, it is crucial to identify not only actions but realistically defined expected results. In fact, when one looks at the international action plans of many municipalities, one finds general performance indicators linked to outputs (i.e., number of activities or participants in an event), but no evidence of the starting point (base line and diagnostic), and fewer references to what has (or has not) been achieved through these actions (outcomes). The same thing happens with the strategic action plans and annual reports of most city networks. Sometimes this is due to the fact that the aims are often formulated too vaguely or are centred on unmeasurable targets. Indeed, goals such as “creating dialogue”, “increasing the city’s influence” or “building lasting relationships” do not permit determination of what should be measured, the frequency of measurement, or definition of what would constitute success.
Hence, though it may seem obvious, it bears repeating that one of the most basic elements of any evaluation process is determining what is to be measured and why (i.e., clarifying goals and targets). This implies spelling out what city networks are seeking to achieve in precise and unambiguous terms. It also means asking the right questions. How will we know that we’ve achieved value locally? What would failure look like? How and to whom will we account for the benefits/damages of participation? (Allemeier, 2018)
The reasons for measuring and holding networks accountable can be remarkably heterogeneous. One of the most evident, is to show the value of the network to its membership (i.e., to keep the current members and attract new ones), but also to external audiences (i.e., international organizations with which to partner, potential donors, etc.). Another reason is to monitor and evaluate the progress made in some of the initiatives promoted by the organization. In this sense, one should be aware that the distinctive nature of some activities may not allow rigorous measurement of their impact. Advocacy, for instance, is not a linear process, and its results are not always straightforward. A further goal could be to generate impact or credibility for funding, which may be a crucial aspect of the network’s revenue model. Last but not least, networks need to be held accountable not only to their constituencies but also to citizens. Strengthening communication and the connection with citizens is vital for legitimizing the services of city networks and for gaining mainstream recognition.
c. Metrics and KPIs: qualitative or quantitative approaches?
There appears to be an overall trend towards measuring the impact of city networks using quantitative methods, which emphasize objective measurements and numerical analysis of data. Nevertheless, indicators like the number of badges given to mayors entering the United Nations (UN) premises, the size of a delegation, or the total number of organized activities may tell us little about the impact of such efforts if the aim is to assess whether the interests and needs of cities and their citizens have been transmitted to the UN and taken into consideration. One way forward, in this regard, might be to inquire into how diplomatic corps worldwide report their work and the tools they are using, which may show the way to more qualitative approaches.
As pointed out by Robert Muggah,5 when defining key performance indicators (KPIs), it might be useful to work with a combination of subjective and objective data. Moreover, it seems appropriate to parse the different types of data generated by city networks and their members, as well as the collection methods. For instance, some data might be self-administered by the city networks’ secretariats (i.e., data collected through surveys to members), while other data would require the collaboration of external agents (i.e., consulting firms). In addition, although all networks regularly reach out to their members, prompting them to provide specific urban data, in most cases sharing this information is a voluntary exercise. Some exceptions here include C40, which makes it compulsory for its members to participate in reporting frameworks like the Global Protocol for Community-Scale Greenhouse Gas Emission Inventories (GPC), the Carbon Disclosure Project (CDP), and the Global Covenant of Mayors’ Common Reporting Framework (CRF).
Another major challenge when measuring the impact of city networking and diplomacy is the lack of a common set of KPIs used by all city networks and a standardized measurement method. This makes comparison very difficult, if not impossible. One way forward could be to look into the KPIs traditionally used by member associations, or to adapt metrics from other disciplines and apply them to the performance of city networks. For instance, in the field of public diplomacy, a number of excellent analyses published in this area (for instance: Hicks, 2021; Buhmann and Sommerfeldt, 2020; or Brown, 2017) could be a valuable source of inspiration.
d. Getting attribution right
Attribution is one of the crucial dimensions of evaluation processes. It involves drawing a causal link and explaining the relationship between observed changes and specific interventions or, in other words, elucidating if, how, and how much a particular diplomatic initiative (undertaken by a particular city network) has caused one specific effect.
Yet, the question of attribution also poses significant challenges. Pahlavi (2007, p. 274) argues that “there are few areas of foreign policy where there is a verifiable relationship between programmes and outcomes”. This might be explained by the fact that there are too many factors involved in achieving goals and influencing outcomes, which “makes any rigid application of a cause-and-effect rationale injudicious” (ibid.).
Following this logic, it must be asked if it is possible for a municipality to connect participation in a particular network with the achievement of specific outcomes in terms of policy influence, knowledge acquisition, or policy implementation. For instance, C40’s 2020 Annual Report states that, thanks to the city-to-city sharing practices the network encouraged, the number of C40 cities incentivizing renewable electricity increased by 650% between 2009 and 2020. While it is plausible to assume that networking and learning spaces such as those provided by C40 might have had a positive effect in cities adopting renewable energy policies, it would be inaccurate to attribute to C40 sole credit for this policy implementation.
5. Conclusions
The increasing number of efforts devoted to analysing the performance and impact of city networks, are testimony to their growing relevance. For decades now, networks have been a crucial asset for cities in their efforts to gain recognition as legitimate stakeholders of the international system. Recognition of urban challenges as part of the global agenda has been fundamental in providing cities with a seat at the global table. Nevertheless, this seat seems to be at the margins of the multilateral system. Many observers argue that the place of cities in the international arena continues to be mainly symbolic and that they lack any real jurisdiction for influencing global policymaking that might have an impact on lived urban realities (Fernández de Losada, 2021).
However, there is consensus among practitioners and analysts that the current ecosystem of city networks has become too complex, that it contains many tensions and contradictions, and that the services on offer are too abundant and dispersed (Fernández de Losada and Abdullah, 2019). Highly professionalized city networks compete to attract the interests of cities, deploying a wide range of activities and services that the end user is not always able to absorb. Although there have been many calls for more and better collaboration schemes between networks (Galceran-Vercher, 2021), the reality shows that shared efforts are still rare. At the same time, few cities approach their participation in city networks strategically and with clear and measurable objectives. Instead, the majority tends to participate in a myriad of organizations.
In this context, showing the value proposition of city networks becomes a key challenge. As management doctrine holds, what cannot be measured cannot be improved (Drucker, 1954). Measuring the impact of the activities performed by city networks could help to focus on the real needs and aspirations of cities, and to improve the services provided in ways that would strengthen local policymaking in key areas. It would also improve accountability and transparency mechanisms and reinforce the legitimacy of city networks among their constituencies and citizens. This is a relevant issue as it would allow cities to justify the strategic decision of joining, remaining in, or leaving a network.
However, measuring the outcomes of the activities performed by city networks is a complex endeavour, especially in the long term. Defining an adequate framework for measuring impact might provide an effective way for city networks to improve their performance and demonstrate the added value of their services. The growing number of city networks and the overlap and duplication between the services that many of them provide is making such a framework particularly necessary.
This briefing has formulated some preliminary guiding principles for elaborating a framework to measure the impact of city networks and the challenges this task presents. However, further debate among practitioners and scholars is needed, with city networks taking the lead.
References
Acuto, M. and Leffel, B. (2021) “Understanding the global ecosystem of city networks”. Urban Studies, Vol. 58(9), pp. 1758-1774
Allemeier, J. (2018) Joining City Networks: Lessons for Impact. Global Policy Opinion, 3 October 2018. Available online at: https://www.globalpolicyjournal.com/blog/ 03/10/2018/joining-city-networks-lessons-impact (last accessed: 5 October 2021).
Brown, K. (2017) “Challenges in Measuring Public Diplomacy”. Center for Public Diplomacy blog. Available online at: www.uscpublicdiplomacy.org/blog/challenges-measuring-public-diplomacy (last accessed: 20 November, 2021)
Buhmann, A. and Sommerfeldt, E.J. (2020) “Pathways for the Future of Evaluation in Public Diplomacy”. CDP Perspectives, Paper 1.
De la Varga, O. (2019) “Entangled: a reflection on the current state of the ecosystem of local government networks”, in Fernández de Losada, A. and Abdullah, H. (2019) Rethinking the Ecosystem of International City Networks: Challenges and Opportunities. Barcelona: CIDOB
Drucker, P. (1954) The Practice of Management. New York: Harper & Row.
Fernández de Losada, A. (2021) “From Multilateralism to Multi-Stakeholder Alliances: Cities Shift from Rhetoric to Politics on the International Stage”, in: Fernández de Losada, A. and Galceran-Vercher, M. (2021) Cities in Global Governance: From Multilateralism to Multistakeholderism? Barcelona: CIDOB
Fernández de Losada, A. and Abdullah, H. (2019) Rethinking the ecosystem of international cityNnetworks. Challenges and opportunities. Barcelona: CIDOB
Foster, S. and Swiney, C. (2021) “City networks and the glocalization of urban governance” in Helmut, A., Nijman, J. and Marcenko, M. (2021) Research Handbook on International Law and Cities. London: Edwar Elgar Publishing
Galceran-Vercher, M. (2021) “The role of cities in a reformed UN: towards the institutionalisation of the World Assembly of Local and Regional Governments”, in: Fernández de Losada, A. and Galceran-Vercher, M. (2021) Cities in Global Governance: From Multilateralism to Multistakeholderism? Barcelona: CIDOB
Hicks, J. (2021) “Defining and Measuring Diplomatic Influence”. K4K Knowledge, Evidence and Learning for Development, 4 February, 2021. Available online at: https://opendocs.ids.ac.uk/opendocs/bitstream/handle/20.500.12413/16476/950_Defining_Measuring_Diplomatic_Influence.pdf;jsessionid=B24F885BEA706EF80C75F6E340C3F313?sequence=1
Kihlgren Grandi, L. (2020) City Diplomacy. London: Palgrave Macmillan
Oomen, B. (2020) “Decoupling and Teaming Up: The Rise and Proliferation of Transnational Municipal Networks in the Field of Migration”. International Migration Review, Vol. 54 (3), pp. 913-939
Pahlavi, P. (2007) “Evaluating Public Diplomacy Programmes”. The Hague Journal of Diplomacy, 2 (3), pp. 255-281
Papin, M. (2019) “Transnational municipal networks: Harbingers of innovation for global adaptation governance”. International Environmental Agreements: Politics, Law and Economics, Springer, Vol. 19 (4), pp. 467-483
Rashidi, K. and Patt, A. (2018) “Subsistence over symbolism: the role of transnational municipal networks on cities climate policy innovation and adoption”. Mitigation and Adaptation Strategies for Global Change, Vol. 23, pp. 507-523
Notes:
1-For a relevant discussion on city networks and the glocalization of urban governance, see Foster and Swiney, 2021.
2-In this briefing we are looking at city networks whose members are mainly local governments.
3-Point raised by the representative of C40 in the seminar.
4-Point raised by the city official of Johannesburg in the seminar.
5-Contribution to the seminar
Authors:
Marta Galceran-Vercher, part-time lecturer in International Relations, Pompeu Fabra University
Agustí Fernández de Losada, senior research fellow and director, Global Cities Programme, CIDOB
Octavi de la Varga, Secretary General, Metropolis, World Association of the Major Metropolises