Stripe Review of Social Sciences in the CGIAR PDF
Preview Stripe Review of Social Sciences in the CGIAR
Stripe Review of Social Sciences in the CGIAR October 2009 Consultative Group on International Agricultural Research Stripe Review of Social Sciences in the CGIAR Review Panel: Christopher B. Barrett (Chair) Arun Agrawal Oliver T. Coomes Jean-Philippe Platteau OCTOBER 2009 i The Science Council of the CGIAR encourages fair use of this material provided proper citation is made. Correct citation: CGIAR Science Council (2009) Stripe Review of Social Sciences in the CGIAR. Rome, Italy: Science Council Secretariat ii Science Council Commentary on the Stripe Review of Social Sciences in the CGIAR October 2009 The impetus for the Stripe Review on Social Sciences in the CGIAR arose from repeated concerns expressed in many EPMRs that social science capacity in Centers was deteriorating. Accordingly, the SC commissioned an initial scoping paper in 2008 that presented the justification for conducting an in‐depth review. The review Panel’s mandate was to provide a System‐level assessment of social sciences in the CGIAR including its quality, coherence, relevance, productivity and capacity, and to make recommendations for improvement. The main phase of the Review was implemented from January to August, 2009. The key elements were: collecting comprehensive data on social science activities and results; formulating a normative framework of “optimal” social sciences roles in the CGIAR against which the current state could be assessed; e‐consultation with Center social scientists to prioritize issues and define hypotheses; analysis of publications and their citations, with a review of selected publications and projects; interviews with staff and partners at nine Center headquarters or regional offices; and soliciting feedback from Centers and external experts on the draft report. The approach taken provides “good practice” for future studies of a similar nature. With one exception, all Centers and CPs were responsive to nearly all the Panel’s requests.1 The Science Council discussed the Panel’s report at its 12th meeting held at CIFOR in Bogor, Indonesia. The Chairman of the Panel, Professor Chris Barrett, presented the report. The Science Council conveys it’s thanks to the Panel Chair and the three members, Drs Arun Agrawal, Oliver Coomes and Jean‐Philippe Platteau, for a frank and insightful report that presents a compelling analysis of the current state of social science in the System. The SC is impressed by the volume of information the Panel assembled to support its analysis including the feedback it solicited from Center staff and social scientists outside the System. The SC emphasizes that this is an important and timely report that deserves full attention from Center management and CGIAR membership to ensure serious consideration of its findings and recommendations. Key Diagnosis The Panel forms its main diagnosis around what it has termed the “new business model”. This model is characterized by: a) a shift from largely unrestricted core funding to relying excessively on restricted funding, most of which comes through very small, short‐duration development‐ oriented grants; b) a lack of vision on what CGIAR social science should be doing that has led to an almost unlimited expansion of social scientists’ agenda; and c) the Centers’ desire to maintain a large cadre of international staff in relation to the work carried out. This has led to a loss of focus and fragmentation in social science work, an erosion of the quality and effectiveness of social science research, a fall of staff morale, and difficulties recruiting and retaining high quality 1 IITA opted not to participate in this review. iii staff. The strategy of maintaining a large cadre of internationally recruited social science staff represents a high‐cost approach for the work being done, much of which could be carried out by well‐trained MS degree holders. The Science Council agrees with this core analysis which reveals shortcomings in Centers’ ability to plan coherent research agendas based on clear strategies and prioritization and defend these agendas in their negotiations with the donors thereby resisting funding‐driven fragmentation. The SC observes that this diagnosis signals broader problems within the System that triggered the CGIAR change management process. The effects of the trends described in the Panel’s report, therefore, are likely to go well beyond social sciences. Nonetheless, the problem may be more serious for social science, than for other disciplines, if the grant money is more fragmented and developmental in orientation and if labor markets for social scientist recruitment are more competitive. These problems may also reflect an inability of the discipline to explain and demonstrate its value in conducting more long‐term and strategic research —both potential and actual—to Center management and donors, and the demand for social science research, therefore, has not been clearly articulated in the Center’s strategic planning. The CGIAR change management process is intended to reverse these trends and improve the coherence and relevance of the CGIAR’s research and the sustainability of long‐term funding. It is imperative that the CGIAR System, including its Centers and donors, take the Stripe Review Panel’s analysis seriously. Special attention to improving social science capacity is needed as part of the change management process. Quality of research The Panel assessed quality primarily by looking at the quality of staff and the quality and impacts of publications. It also reviewed on‐going projects that shed light on the scope and role of social sciences, project funding and the planned partnerships. Close analyses of a sample of projects allowed assessment of methodological quality and the match between objectives and approaches. The CGIAR System should be extremely concerned by the findings that social science research is judged to be methodologically weak and declining in quality. In addition, there appears to be a lack of critical mass of high quality social scientists with sufficiently diverse disciplinary skills. The Panel also found that partnerships are increasingly transactional (even competitive in the case of NARS) and the quality of the partnerships with ARIs is weak. Another aspect of declining quality is that data collection, management and exploitation were found to be deficient. The SC strongly endorses the need to systematically improve the management of data and its use internally and externally, including attention to ethical issues in data collection. Nonetheless, the SC emphasizes that the review has also revealed pockets of excellence in the CGIAR. The Panel notes that excellence seems to be associated with: sufficient seniority among the researchers, clearly defined long‐term projects with assured funding, long‐term partnerships with ARIs, and inclusion of an inter‐disciplinary component. Center management can influence many of these factors despite funding pressures. iv In its assessment of quality, the Panel applied criteria used in academia, such as number of publications and citation. The SC accepts that these criteria are also applicable to a highly mission‐driven organization. It emphasizes, however, that publications and citations as used in the profession at large is too narrow a perspective. In the CGIAR social science research ambition needs to also include a strong commitment to solving important problems that impact poor people. Comparative advantage and relevance The report lists the attributes that give social scientists in the Centers a comparative advantage in generating international public goods. The most important attribute is the ability to organize research around problems rather than disciplines—the ease of conducting inter‐disciplinary research—and work at an international scale and sustained over time. Other elements of comparative advantage include the Centers’ physical infrastructure and long‐term presence in developing countries, and a cadre of well‐trained, internationally recruited staff. Given comparative advantage, the Panel identified three areas where it sees the greatest opportunities for CGIAR social science research to be effective in serving its mission: (i) technological change that sustainably increases agricultural productivity by and for the poor; (ii) natural resource conservation that benefits the rural poor directly or indirectly via environmental services, especially those that help agricultural productivity; and (iii) institutional innovations and policies that enhance the quality of life for poor and marginalized agrarian populations. Of these, the Panel considers the first as the single most important area of CGIAR social science comparative advantage because it most fully capitalizes on the opportunity for close interaction between social and biophysical scientists that is distinctive to the CGIAR. In the Panel’s view, a plurality if not a majority of staff should be in the first area. The Panel hypothesizes that, as Centers’ missions have expanded, the most competent social scientists have been drawn into the marketing and policy area, with fewer engaged in research on productivity enhancement. The SC agrees with the Panel’s judgment that Center management needs to take action to protect the focus and relevance of the Center’s research agenda. Drifting with funding trends cannot be fully avoided but it can be sharply reduced. However, in order to improve the overall relevance, measures at the System level among the donor community are also clearly needed. For example, critical long‐term system studies using sentinel sites would require commitment from donors to fund such work and commitment from scientists and management to engage in long‐term work to accumulate data, comparisons and experiences. Donors and Center management need to understand better what good social science research can do for the CGIAR and for development more broadly. Impact What ultimately determines the relevance of CGIAR’s social science is its ability to generate impact. The SC finds in the Panel’s discussion of impact three distinct dimensions that are not clearly separated in the report: i) impact on long‐term development goals from CGIAR’s research; ii) impact of social sciences on influencing research priorities; and iii) impact evaluation of research as a component of the social scientists’ research agenda. The SC would have liked to see much clearer analysis of the demand for and value of social science research in the context of enhancing impact of the CGIAR’s research. Examples of v strategic outputs from social science in different areas that very likely have enhanced the likelihood of development impact would have been helpful. However, the pathway from social science research quality to impacts on CGIAR goals is very indirect and highly uncertain. The Panel observes that social science research can most directly influence the likelihood of impact through improved research priority setting and by influencing policies. The SC also notes that in the past attempts to use a systematic ex ante analysis to determine priorities have often not been very successful. It would be important to trace the influence of ex ante priority setting research on what have eventually turned out to be successful choices in terms of real impact generated. Regarding the third aspect of impacts, the Panel identified an “impact measurement obsession”. According to the Panel, the number of ex post impact assessments (EPIAs) has proliferated, and these are methodologically weak and being conducted on small projects (rather than programs) too soon after termination. The Panel suspects that this trend has been fuelled by the donors’ desire to report back to their constituencies on effective use of funds, and making all research projects accountable for showing impact, despite the well‐known uncertainties of research payoffs. At the same time, the Centers are responding to a requirement of the CGIAR’s performance measurement system (PMS) to conduct a certain number of EPIAs annually. The Panel acknowledges and the SC emphasizes that SC/SPIA has strongly pushed for the PMS to require a limited number of high quality EPIAs to be conducted on large research efforts and sufficiently long after research has been completed for increasing the ability of the studies to detect and credibly document sustainable impacts. Confirming the Panel’s observations, SC/SPIA has observed that a very large proportion of studies submitted each year to the PMS do not qualify as EPIAs at all but are end‐of‐project evaluations. Recommendations The Panel provides four broadly‐based recommendations, which cover management reforms, re‐ organization and re‐focus in social sciences, updating social science staff management practices, and fostering a culture of rigorous social science research. The SC has the following comments on the recommendations and suggestions on how they might be implemented at the System and Center level. Recommendation 1: Undertake essential management reforms 1a) Resolve the mismatch between the business model and staffing patterns 1b) Realign management incentives 1c) Improve leadership selection 1d) Tighten the focus on comparative advantage 1e) Focus on impact but end the impact measurement obsession 1f) Mainstream gender equity as a basic axiom of CGIAR research 1g) Require full indirect cost recovery The SC agrees in principle with these recommendations. The CGIAR reform process promises to address many of the problems underlying this recommendation (especially components 1a and 1d) by improving the coherence of the overall CGIAR program and the commitment of core funding. Implementation of this recommendation should, however, not wait for nor solely rely on the change process. There are important steps Centers can take in the meantime such as vi improving their own strategic and medium‐term planning to guide the inclusion or exclusion of activities and choice of research programs and the roles of social science within them (as strong Centers are already doing). Components 1b, 1c and 1g are clearly the immediate responsibility of individual Center management with Board oversight but should also be pursued by the Consortium for affecting change more uniformly across Centers and programs. Component 1e on impacts can be best addressed through appropriate design of Mega‐programs and agreement on the outcome targets and steps for generating impacts that will be monitored in a systematic fashion. Component 1f on mainstreaming gender addresses a concern that has been identified in several reviews and needs to be appropriately addressed at the System level. Recommendation 2: Re‐organize and re‐focus CGIAR Social Science 2a) Restore longer‐term partnerships, especially upstream 2b) Focus training and capacity building on research mentoring 2c) Organize a Regional Systems Analysis Mega‐Program 2d) Organize a Mega‐Program on Stimulating and Evaluating Innovations 2e) Shrink unproductive social science units The Science Council sees merit in what is being recommended, but suggests that the implementation requires further discussion. Addressing recommendations 2a and 2b will be an important challenge in moving to a programmatic mode in research implementation that will increasingly involve partners outside the System. In its response to the recommendations to establish Mega‐programs on particular subjects, the SC interprets this to mean that, rather than Mega‐programs per se, a systematic way of organizing research units or platforms for addressing research topics across Centers should be formed. The elaboration in the report related to these components (2c and 2d) should be incorporated into the design of future Mega‐programs, providing important roles for social science in these programs. Recommendation 3: Update social science personnel management practices 3a) Introduce a CGIAR Young Scientists Program 3b) Increase entry‐level compensation packages 3c) Establish a clear research career track for social scientists 3d) Restore competitive travel and sabbatical programs 3e) Employ modern human resources management practices This cluster of recommendations is intended to address several shortcomings hampering recruitment and retention of good quality social scientists in the CGIAR. The SC appreciates that the Center managers are aware of many of these issues and of the importance to engage well‐ trained and highly motivated young scientists from sufficiently diverse disciplinary background. Recommendations 3a and 3b would help but other measures will also be needed. The CGIAR would need additional funding for implementing 3a to re‐establish a post doctoral program. Even then it would be important to consider ways in which the new program can be designed to meet emerging needs. vii The SC considers it important that the Centers restore their attractiveness as a long‐term employment option for high quality scientists by exploring a range of both full and part time (shared) appointments, and maintaining locational flexibility. This depends not only on human resource management but also on the excitement that the System’s mission can generate. The implementation of recommendations 3b‐3e must be tailored to the highly variable circumstances and conditions at each Center. A more uniform approach can only be led from a central unit that would determine human resources policies and practices across Centers. Even then, location‐ specific circumstances need to be taken into consideration. Recommendation 4: Foster a culture of rigorous social science research 4a) Restore social science research seminar programs 4b) Individual performance measurement 4c) Stop wasting money on in‐house publications other than policy or research briefs 4d) Establish a CGIAR Institutional Review Board In this recommendation, components 4a and 4b propose readily supportable and practical actions to stimulate motivation and quality. The SC believes, however, that while dissemination of research results (4c) needs to be cost‐ and time‐efficient, strategies for publications and other means of dissemination need to consider different target groups and multiple outlets may be necessary for disseminating the same results. The Panel’s recommendation on Institutional Review Board echoes the recommendations from the SC’s recent studies on ethics that suggested it is imperative for scientists to have their research plans approved by a review board. This is particularly important for research that involves human subjects, and the SC again flags this need for Center Boards and the Consortium Board. Conclusions and next steps The main messages in this report and its recommendations in the context of a much changed research and development environment pose new challenges for research, involving social sciences, as well as research management. The SC fully agrees with the Panel’s optimism that there are now excellent opportunities for building strengthened and relevant social science components into the CGIAR. The causes for this optimism include the renewed recognition among donors and governments of the importance of agriculture and agricultural research to development; renewed interest in applied development research in the social sciences; and the CGIAR reform process itself. The SC strongly recommends that the System and the Centers collectively and without delay put in place a process to further discuss and implement the very timely and relevant recommendations of this report. This must involve further discussion with Center managers as well as with social scientists themselves. Some of the recommendations could be implemented at relatively low cost, such as revamping recruitment processes to attract higher quality social scientists. Others will need sustained additional funding. Fortunately, there are indications that some donors would be interested in providing such support if the System is ready to move forward. viii Christopher B. Barrett Stephen B. and Janice G. Ashley Professor of Department of Applied Economics and Management Applied Economics and Management International Professor of Agriculture Co‐Director, African Food Security & Natural Resources Management Program Cornell Institute for International Food, Agriculture and Development 315 Warren Hall http://ci ifad.cornell.edu/ Ithaca, NY 14853‐7801 USA Telephone: 607‐255‐4489 Fax: 607‐255‐9984 Associa te Director for Economic Development Programs E‐mail: [email protected] Cornell Center for a Sustainable Future http://aem.cornell.edu/faculty_sites/cbb2/ http://sustainablefuture.cornell.edu/ November 20, 2009 Dr. Rudy Rabbinge Chair, Science Council Consultative Group on International Agricultural Research c/o FAO Viale delle Terme di Caracalla 00153 Rome, Italy Dear Dr. Rabbinge, Attached please find the report of the Social Science Stripe Review (SSSR) panel that the Science Council appointed late last year. The other three panel members – Professors Agrawal, Coomes and Platteau – and I thank you and the rest of the Science Council for granting us this opportunity to learn about the wide‐ranging work of social scientists within the Consultative Group on International Agricultural Research (CGIAR) and, we hope, to offer constructive insights on the strengths, weaknesses and potential of the social sciences to serve the broader CGIAR mission. Per the terms of reference, our report offers a comprehensive assessment of CGIAR social science research followed by a set of clear recommendations based on the analyses in the assessment. An undertaking of this scale and scope requires considerable skilled support. The panel’s work was aided invaluably by a large number of people scattered throughout the world. We especially thank Dr. Sirkka Immonen of the Science Council Secretariat, who expertly coordinated the panel’s activities, data collection and interaction with Centers and Challenge Programs, and provided insightful Cornell University is an equal opportunity affirmative action educator and employer. ix
Description:The list of books you might like
