Ensuring impact: using methods and metrics to assess results
October 2015 — It's a no-brainer: If you want to measure business success, look to the numbers. But with the advent of hybridized "philanthrocapitalism" and technologies enabling the measurement and analysis of practically everything, the same standard has become prevalent in the social sector as well. Giving a clear, quantifiable picture of a program's impact helps beneficiaries and funders alike by ensuring that grant money reaches its target, and helping foundations, governments, and NGOs make better funding choices.
This month, URB.im looks at innovations in quantification from cities around the Global South — programs and practices that give social success a much-needed metric and enable funders to be guided by strategy and statistics instead of sentiment. Follow the conversation as it unfolds over the course of the month and share your thoughts in the comments below.
EL Calvario Cultivo Social (CCS) busca obtener fondos como varias de las propuestas que buscan proponer la ayuda de técnicos para la mejora urbano en los barrios de Caracas. Estos fondos, ya se sabe, cada día están mas restringidos por los requisitos del mundo filantrópico y cada día mas difícil por la falta de capacidad en conllevar el propicio seguimiento y monitoreo a los proyectos y adecuarse a los mecanismos occidentales e científicos que busca medir el supuesto éxito de propuestas llanas pero ingeniosas.
چرا توصیههایپانزده سال بعد: ارزیابی پروژه نواب
MIS – Integrasi informasi untuk mengukur Dampak Program
System MIS online dirancang dengan menggandeng Farsight dalam desainnya dan telah dilaksanakan setahun terakhir. Dalam struktur PNPM Peduli, TAF menggandeng Mitra Payung sebagai EO (executing organization) dan Mitra Kerja dalam hal ini adalah CSO. Terdapat Tujuh EO dan 76 CSO yang bekerja dalam 6 isu utama yaitu: anak dan remaja rentan, masyarakat adat terpencil, disabilitas, diskriminasi agama, waria, pelanggaran HAM berat. EO dan CSO diwajibkan mengisi MIS online secara berkala sebagai bagian dari proses monitoring dan evaluasi sehingga ada umpan balik terhadap penilaian kinerja dan keberhasilan proyek. Sementara itu proses pembelajaran disini dilakukan melalui pendokumentasian praktik-praktik baik dan perubahan signifikan yang terjadi selama implementasi proyek.
Assessing malaria control programmes in Accra
A public health triangulation approach was used for this study to assess the malaria burden in the major cities and also to better understand the implications of these findings for malaria control in Ghana. The approach involves the process for reviewing, synthesizing, and interpreting secondary data from multiple sources that inform public health decisions. The major sources of data included household surveys, both those with a nationwide sample and those focused exclusively on Accra, data reported monthly by health facilities in Ghana, and reports of previous studies documenting the burden of urban malaria in specific communities. This provided a means of validating results to increase credibility in ensuring a more detailed understanding of the findings.
Measuring impact of education reform
El observatorio ambiental de Bogotá
Realizando os objetivos de desenvolvimento em Curitiba
O trabalho da ASID começa com um estudo crucial: a análise do cenário. Para isso, é aplicado o IDEE, Índice de Desenvolvimento da Educação Especial, uma ferramenta de metodologia própria que cruza 26 indicadores e 143 itens de avaliação divididos em três áreas – Gestão, Infraestrutura e Captação de Recursos – e mostra de maneira clara os problemas enfrentados e as principais potencialidades. Após a análise criteriosa das informações – com auxílio de líderes de cada entidade –, é apresentado um amplo diagnóstico e um plano estratégico de ação, elaborado em oito esferas: impacto social, recursos humanos, gestão jurídica, gestão financeira, captação, marketing, liderança e estrutura física.
City BRT, neighbourhood BRT, and the "Corridors of Freedom": Image and reality in Johannesburg
Creative metrics for diverse portfolios
Viviendas mejoran la calidad de vida de los habitantes marginados
En este sentido, el Abdul Latif Jameel Poverty Action Lab (J-PAL) de Massachusetts Institute of Technology, surgió en 2003 como una red de profesores afiliados de todo el mundo (que hoy en día cuenta con 127 miembros). Su misión es reducir la pobreza garantizando que las políticas públicas estén informadas por evidencia científica, y que la evidencia se traduzca en acción. Realiza evaluaciones aleatorias del impacto para probar y mejorar la eficacia de los programas; y hace análisis de costo-efectividad trabajando con gobiernos, ONGs, y organizaciones internacionales de desarrollo.
Measuring informality in Nairobi's Jua Kali
For many Nairobians the 'hustle' is part of day to day life: individuals engage in small-scale manufacturing and artisan trade; they provide transport services; and are the frontline innovators fueling the growth in the information, communication and technology sector.
Trung tâm Hỗ trợ Phát triển Cộng đồng LIN: Tổ chức Hỗ trợ Đánh giá Đắc lực
10 bộ công cụ cho các NPOs địa phương
O trabalho da Brazil Foundation começa com uma seleção anual – por meio de edital – de iniciativas que proponham soluções criativas para desafios enfrentados em todo o país nas áreas de Educação, Saúde, Direitos Humanos, Cidadania e Cultura. Para serem válidas, as propostas inscritas devem ser inovadoras, ter metodologia clara e eficaz, apoio local e potencial para influenciar políticas públicas. Os pré-requisitos são investigados de perto por analistas, que visitam cada um dos requerentes, conferem sua legitimidade e mesuram qualitativa e quantitativamente seus frutos, projetando passos futuros.
الصندوق الاجتماعي للتنمية: يغير المجتمع ويُثبت تأثيره
El objetivo de la iniciativa Cali Como Vamos es el realizar un seguimiento detallado a la evolución de lo que ellos denominan la "calidad de vida" en la ciudad. Uno de los aspectos más innovadores de este proyecto es que no solo utiliza una serie de indicadores para la elaboración de sus informes, sino que además realiza anualmente una amplia encuesta de opinión pública. Esto permite abrir el proceso de evaluación y de formulación de políticas públicas a la ciudadanía, y además añade un componente muy importante a la valoración del impacto de los programas y las políticas públicas, sobre todo en ámbitos en los que éste no se puede medir de forma efectiva mediante el uso de indicadores.
CESA is a public policy think-tank and nonprofit organization dedicated to evaluating the economic impact of policies in Nigeria. It was established to "advocate for greater fiscal transparency and accountability, reduction in leakages of public funds and improvements in governments' delivery of social and public services" through evidence-based research. The Abuja-based organization focuses its research and evaluation on seven major areas, one of which is Program Evaluation, Poverty Measurement and Analysis (PEPMA). This arm of research evaluates the effectiveness of various programs, policies and projects aimed at addressing development challenges. PEPMA has two main elements – Program Evaluation, and Poverty Measurement and Analysis – and cuts across a number of areas such as gender disparity, social inclusion and inequality.
Mumbai-based Vera Solutions believes that data can transform the efforts of social-change organizations. The company uses mobile data and cloud-based systems to build customized databases. These databases can be automatically analyzed and give important programmatic information to staff in home offices and in the field to understand quickly the effectiveness of particular approaches and programs. Time, effort and funds can be spent more efficiently and effectively with easily accessible numbers from the field.
মাইক্রো ফ্র্যাঞ্চাইজিং : দুগ্ধজাত খাদ্যে বিষক্রিয়া রোধে ক্ষুদ্র অথচ কার্যকর সমাধান
মাইক্রো ফ্র্যাঞ্চাইজিং তুলনামূলক নতুন হলেও সফল একটি উদ্যোগ হিসেবে পরিগণিত হচ্ছে। অন্য সকল ফ্র্যাঞ্চাইজির মতই এটির কার্যক্রম শুরু হয়, ধরা যাক, পিজা হাট। কিন্তু ব্যতিক্রম হচ্ছে, মাইক্রো ফ্র্যাঞ্চাইজি সামাজিক পরিবর্তনের সূচনা করে। এই লেখায় আলোকপাত করা হবে এমনি একটি মাইক্রো ফ্র্যাঞ্চাইজির ওপর, যা বর্তমানে চলমান। কৃষি উৎস, কেয়ার বাংলাদেশ যার পথিকৃৎ। ঢাকা মহানগরীর অন্যান্য সামাজিক ব্যবসা সংস্থাগুলোর সাপেক্ষে কৃষি উৎস বৈচিত্রময় কারণ কোনপ্রকার অতিরিক্ত পদক্ষেপ ছাড়াই এটি ক্রমবিকাশমান এবং সামাজিক পরিবর্তনে এর প্রভাব পরিসংখ্যানের মাধ্যমে পরিমাণযোগ্য।
Comments
Enviromental conservation
I have been fascinated with the environmental conservation in Abuja.Hence iwish authorities in my town could do the same.
From Mangochi town,
Malawi southern Africa.
Can good data save lives?
Hello Carlin,
Great piece! I think, as per Vera solutions, data really saves lives, since M&E now is part and parcel of the project management cycle. When numbers and data is correct, we will have a valid baseline information which will lead to valid and credible final measurements that lead to more knowledge on the efficiency of any project which was mentioned as part of your article and component of the Vera Solutions mission as I believe.
Implement a regional office to measure business success
In each region set up an office that lists the various investment sectors and have the results in each sectors every 3 years .
ciudadanos participativos
Jorge, muy interesante el ejemplo de Cali sobre la encuesta y la herramienta del reporte para dar seguimiento a las acciones del municipio. Sería muy interesante identificar como impacta esta información en las decisiones de la ciudadanía ante la elección de la siguiente administración municipal. Definitivamente mientras más información tenga el ciudadano y de seguimiento a los compromisos del municipio, podrá diferir de un buen gobierno y exigir lo que la ciudad requiere.
Measuring informality in Naiorbi's Jua Kali
Hello Hilary!
Great post! Quantifying the contribution of informal labourers will hopefully produce results which can be used as a counter argument to the variety of threats the workers face daily. Highlighting the way that informal labor is viewed is extremely important as more people who benefit from the convenience of the local botanist or service provider will be more inclined to support informal workers. Hope the work is a success.
Visibility for Informal Workers
Ashali, you bring up a great point here: that measuring and counting is not just about validating projects and programs but also about bringing visibility to certain issues and groups. The example of informal workers is an interesting one, and I like how you point out that people who use the services of informal workers would be more likely to recognize and acknowledge them if we can better show who they are and how they contribute to cities. Good statistics also really help in moving forward government policy and planning to support the needs of informal workers, because they often don't know about certain worker groups--take domestic workers, for example, who are often inside homes and quite invisible. Without the numbers to really tell the story of the needs of informal workers, it's difficult to get the right policies and practices in place.
re: Measuring informality in Nairobi's Jua Kali
Thank you for your comment Ashali. I agree it is really important to begin to capture and quantify their contribution as they are often painted negatively by the media and government. This is especially true with local officials often profit from extorting these same individuals. This places informal business owners in a precarious situation where they must pay the bribe to continue to safely operate, yet at the same time the loss of these monies is both a hit to the business and a prohibitive factor that makes it difficult for them should they desire to begin participating in the formal economy. It is a catch-22 for many and hopefully the ability to demonstrate the real and substantial impact these individuals and business have on the countries GDP will promote a more equitable and egalitarian method for adapting formal systems to meet the real needs and demands of a rapidly growing economy.
Measuring Educational Reform in Lagos
Hey Olatawura, I found your article on the Ecko program in Lagos schools very interesting. I would be interested to learn more about the capacity development learning undertaken with the teachers and the impact this has over time on students performance. It is also very encouraging to see infrastructure and technical development funded alongside traditional education/learning objectives. It is often the case that donors are more apt to fund innovative ideas than the hard more static necessities. In my research in Kenya, I have found a strong relationship between the quality and availability of adequate sanitation and hygiene availability and girls enrollment and performance in secondary school. Look forward to seeing the results of this project as the WB funding finishes up and the local government takes accountability for these initiatives.
Viviendas mejoran la calidad de vida de los habitantes marginado
Hi María Fernanda,
Thanks for your article. Your piece shows two very important things: how joining forces is indispensable – including looking for help internationally to solve "small" local problems – and how important it is to know the areas and the families that will be (or already are) beneficiaries of a social project, to go to them, to listen to them, to make field research... so you can really implement the work and get effectiveness. It is amazing how much money is invested in social initiatives without the real needs and priorities are even researched. I showed one clear example of this in Brazil in my article on public transportation: http://urb.im/ca1507rjp – it's really sad and hopeless. How many work (and waste of money) could be avoided if we could count on J-PAL's help since the beginning in all social projects of this world... :) I'm glad they're making a difference!
Meassuring Impact
Hi Andrea, definitely meassuring social programs ensures the impact on beneficiaries. In Mexico before 2000 social programs didn't have diagnostic nor evaluation, but nowadays thanks to the international sphere it is more common to invest the monew in programs that really impact life of persons.
Participation- based Poverty Alleviation
Hello Widya, I found your article on Surabaya very interesting. Including the participation of the target population is crucial. "Whatever you do for me but without me, you do against me" said Gandhi. In Medellin, I see many urban projects that seem great, and that aim at reducing poverty. Sometimes though, I am not sure that they answer the major needs of the population. There also seem to be a great participation of the local participation. However, many times it seems to me that what results from the participative process is not translated in governmental decision-making, which makes participation uneffective, or even "faked"... It is great if in the case you mention, it has a real impact on policy making.
Cali Como Vamos
Hola Jorge, a mí también me parece muy interesante la red de ciudades como vamos. Fui a la socialización de la encuesta de percepción de Medellín Como Vamos y me alegró ver que vinieron muchas personas de la alcaldía y de las empresas más importantes de la ciudad (el metro, EPM…). Me pareció que sí podía tener un efecto directo sobre el diseño de las políticas públicas. La satisfacción con el Metro bajó, por el caos de las horas pico, y la empresa anunció la compra de nuevos trenes… También en el contexto colombiano, donde la corrupción es muy presente, me parece una herramienta interesante para asegurarse del buen uso del dinero público. Los ciudadanos esperan resultados medibles. En Medellín, el marketing urbano es excelente...demasiado bueno… Esta herramienta permite ver si las políticas públicas tienen un impacto más allá de toda la propaganda…
Add new comment