Grid Electricity Expansion in Tanzania by MCC: Findings from a Rigorous Impact Evaluation

368 Pages·2017·3.27 MB·English

Grid Electricity Expansion in Tanzania by
MCC: Findings from a Rigorous Impact
Evaluation
February 24, 2017

Duncan Chaplin, Arif Mamun, Ali Protik, John Schurrer, Divya Vohra,
Kristine Bos, Hannah Burak, Laura Meyer, Anca Dumitrescu,
Christopher Ksoll, and Thomas Cook
Submitted to:
Millennium Challenge Corporation
1099 14th Street, NW
Suite 700
Washington, DC 20005
Project Officers: Jennifer Heintz and Shreena Patel
Contract Number: MCC-10-0114-CON-20-TO05
Submitted by:
Mathematica Policy Research
1100 1st Street, NE
12th Floor
Washington, DC 20002-4221
Telephone: [phone]
Facsimile: [phone]
Project Directors: Arif Mamun and Duncan Chaplin
Reference Number: 06919.431
This page has been left blank for double-sided copying.
ACKNOWLEDGMENTS
This report is the culmination of an 8-year project and would not have been possible without
the dedicated efforts of staff at the Millennium Challenge Corporation who initiated, supported,
and helped to guide this research activity. Key players included Jennifer Heintz, Shreena Patel,
John “Jack” Molyneaux, Jennifer Sturdy, Laura Rudert, and Anne Rothbaum Pizer. We also
thank Karl Fickenscher and Mathew Kavanagh from the MCC’s Resident Country Mission in
Tanzania, as well as Himesh Dhungel and Kate Iovanna from the MCC energy sector team.
Host nationals from the Millennium Challenge Account-Tanzania (MCA-T) and the
Tanzanian Electric Supply Company (TANESCO) played major roles participating in,
cooperating with, and supporting our efforts. At MCA-T this included Bernard Mchomvu, Issac
Chanji, Chedaiwe Luhindi, Peter Kigadye, Athanas Alois, Ahmed Rashid, Paschal Assey, Joseph
Hayuni, Salum Ramadhani, Florence Gwang'ombe, Reginald Ndindagi, William Christian, and
Deborah Sungusia. We also appreciated the participation and cooperation of TANESCO staff.
Two firms collected the bulk of the data analyzed in this report. This report would not have
been possible without the resourceful data collection implemented by staff at NRECA who
collected the baseline data. We especially thank James Walsh, ATM Selim, Hussain Samad,
Sylvester Ngallaba, and Saidi Nyambaya. We also sincerely appreciate the hard work of their
team of interviewers, as well as the community leaders, households and businesses that
participated in the surveys. At follow-up, EDI did an exemplary job finding our baseline sample,
enhancing our survey instrument, and producing a high-quality dataset. Our primary contact was
Amy Kahn. She was joined by a team with great depth including Respichius D. Mitti, Deo
Medardi, Bhoke Munanka, Abraham Ngowi, Mark Johnson, and Matthew Wiseman. Joachim de
Weerdt co-founded EDI and helped us make contact with his colleagues.
We also thankfully acknowledge support from the following entities and individuals. Camco
Clean Energy took the lead on overseeing implementation of the Financing Scheme initiative.
Staff from Camco, including Jeff Felten, Regina Fumbuka, Baldwin Msowoya, Rose Komba,
and Cyril Batalia, helped execute the communication campaign to inform community members
about the low-cost connections offered under the financing scheme initiative. Camco staff,
including Andrew Mnzava, also provided the photos used in our report. DHInfrastructure, a firm
with expertise in the energy sector, provided valuable input. Specifically, Denzel Hankinson,
Josh Finn, Lauren Pierce, and Brendan Larkin-Connolly of DHInfrastructure supported us at
various stages of the evaluation. Last, but not least, Leonard Sibomana, our consultant in
Tanzania, acted as our eyes and ears on the ground throughout this project.
This report also benefited greatly from the contributions of our colleagues at Mathematica.
Phil Gleason provided valuable feedback on a draft of this report. We also benefited from
discussions with Ken Fortson, Steve Glazerman, Lesley Hildebrand, Frank Potter, Candace
Miller, Nancy Murray, and Peter Schochet. We would like to thank Elizabeth Kelley, Ebo
Dawson-Andoh, Ji-Hyeun Kwon-Min, and Miles Watkins for excellent research assistance;
Xiaofan Sun, Yuhong Zheng, and Cathy Lu for programming support; Jennifer de Vallance and
Brigitte Tran for help with production; Carol Soble and Amanda Bernhardt for help with editing;
Kimberly Ruffin, Jessica Hegedus, and Colleen Fitts for help with formatting; and Samira
Siddique, Ryan Collins, Lindsay Eckhaus, and Janine O’Donnell for help with project
management. Last, but not least, Jeffrey Smith from the University of Michigan Economics
Department provided very insightful and comprehensive comments throughout this report.

iii
This page has been left blank for double-sided copying.
CONTENTS
ACKNOWLEDGMENTS ................................................................................................................................ iii
LIST OF ACRONYMS AND TERMS .......................................................................................................... xiii
EXECUTIVE SUMMARY ............................................................................................................................. xv
I INTRODUCTION .............................................................................................................................. 1
A. Overview of the energy sector project ....................................................................................... 2
B. Project logic and conceptual framework for the T&D lines and FS initiative ............................. 3
C. Organization of this report ......................................................................................................... 5
II LITERATURE REVIEW.................................................................................................................... 7
A. History of electricity investments in Africa ............................................................................... 10
B. Challenges with expanding access ......................................................................................... 12
C. Barriers to connecting .............................................................................................................. 13
D. Interventions to reduce costs................................................................................................... 15
E. Benefits of electrification .......................................................................................................... 17
III EVALUATION DESIGN.................................................................................................................. 29
A. Evaluation questions ............................................................................................................... 31
B. Impact evaluation design ......................................................................................................... 32
C. Outcomes for impact analysis: Primary and secondary outcomes by domain ........................ 38
IV DATA SOURCES AND ESTIMATION APPROACH ...................................................................... 45
A. Sampling .................................................................................................................................. 47
B. Data collection ......................................................................................................................... 52
C. Estimation approach ................................................................................................................ 61
V IMPACTS OF T&D LINE EXTENSIONS ........................................................................................ 71
Executive summary ........................................................................................................................ 73
Introduction .................................................................................................................................... 77
A. Main findings............................................................................................................................ 78
B. Findings for subgroups ............................................................................................................ 97

v
CONTENTS MATHEMATICA POLICY RESEARCH
VI IMPACTS OF LOW-COST-CONNECTION OFFERS ................................................................... 99
Executive summary ...................................................................................................................... 101
Introduction .................................................................................................................................. 105
A. Main findings.......................................................................................................................... 105
B. Findings for subgroups .......................................................................................................... 116
VII IMPACTS OF ACTUALLY CONNECTING TO THE ELECTRIC GRID ....................................... 119
Executive summary ...................................................................................................................... 121
Introduction .................................................................................................................................. 124
A. Main findings.......................................................................................................................... 125
B. Findings for subgroups .......................................................................................................... 134
C. Robustness of exploratory impacts of connection: Instrumental variables approach ........... 136
D. Within-study comparison ....................................................................................................... 138
VIII SUMMARY AND CONCLUSIONS ............................................................................................... 151
A. Impacts of T&D lines ............................................................................................................. 154
B. FS initiative impacts ............................................................................................................... 156
C. Impacts of actually connecting to the grid ............................................................................. 158
D. Differences between findings from more and less rigorous evaluation designs ................... 161
E. Conclusions ........................................................................................................................... 161
REFERENCES .......................................................................................................................................... 165
APPENDIX A DESCRIPTION OF WEIGHTS ......................................................................................... A.1
APPENDIX B BASELINE EQUIVALENCE TABLES .............................................................................. B.1
APPENDIX C OUTCOME DEFINITIONS ...............................................................................................C.1
APPENDIX D CONVERSION FACTORS ...............................................................................................D.1
APPENDIX E SUPPLEMENTARY TABLES ON ESTIMATED IMPACTS ............................................. E.1
APPENDIX F THE PROBLEM OF INACCURATE BASELINE GPS DATA AND ATTEMPTED
ANALYTIC SOLUTIONS .................................................................................................. F.1
APPENDIX G WITHIN-STUDY COMPARISON DETAILS .................................................................... G.1
APPENDIX H EXPLORATORY ANALYSES ..........................................................................................H.1
APPENDIX I ESTIMATING TOTAL NUMBER OF CONNECTIONS ..................................................... I.1
APPENDIX J SUPPLEMENTARY ANALYSES RELATED TO IMPACTS OF ELECTRICITY
ACCESS ON EMPLOYMENT BY GENDER FOLLOWING DINKELMAN (2011) ........... J.1
APPENDIX K STAKEHOLDER COMMENTS ON DRAFT REPORT ..................................................... K.1

vi
TABLES
ES.1 Impacts of line extensions and low-cost-connection offers for selected outcomes ................ xviii
II.1 Programs to reduce connection costs in Africa ........................................................................ 16
II.2 Summary of research studies on benefits of electrification ...................................................... 19
III.1 Technical approach to impact evaluation ................................................................................. 32
III.2 Community data: Primary and secondary outcomes by domain .............................................. 40
III.3 Household data: Primary and secondary outcomes by domain ............................................... 41
IV.1 Number of intervention and potential comparison communities for the community
survey by region ........................................................................................................................ 48
IV.2 Purpose, target sample sizes, respondents, and timing of baseline and follow-up
surveys for the Tanzania energy sector evaluation .................................................................. 53
IV.3 Community survey respondents for the T&D lines and FS initiative evaluations (target
sample sizes in parentheses) ................................................................................................... 54
IV.4 Listing and pole data ................................................................................................................. 57
IV.5 Sample size and response rates in the follow-up household survey ........................................ 59
IV.6 Baseline characteristics used as control variables for T&D lines and FS initiative
impact analyses ........................................................................................................................ 63
IV.7 Control variables for the exploratory analysis of impacts of being connected to the
national grid .............................................................................................................................. 66
IV.8 Summary of sensitivity analyses using alternative estimation approaches .............................. 69
V.1 Community-level T&D impacts on connection rates ................................................................. 80
V.2 Household-level T&D impacts on connection rates .................................................................. 80
V.3 Household-level T&D impacts on energy use .......................................................................... 83
V.4 Community-level T&D impacts on energy use ......................................................................... 84
V.5 Community-level T&D impacts on education and child time use ............................................. 86
V.6 Household-level T&D impacts on education and child time use .............................................. 86
V.7 Community-level T&D impacts on health and safety ................................................................ 88
V.8 Household-level T&D impacts on health and safety ................................................................. 89
V.9 Community-level T&D impacts on business and adult time use ............................................... 91
V.10 Household-level T&D impacts on business and adult time use................................................ 92
V.11 Community-level T&D impacts on economic well-being ........................................................... 94
V.12 Household-level T&D impacts on economic well-being ............................................................ 95
V.13 Community-level T&D impacts on composition and mobility .................................................... 96

vii
TABLES MATHEMATICA POLICY RESEARCH
V.14 Household-level T&D impacts on composition and mobility ..................................................... 96
VI.1 Estimated FS initiative impacts: Connection rates ................................................................. 107
VI.2 Estimated FS initiative impacts: Energy use .......................................................................... 109
VI.3 Estimated FS initiative impacts: Education and child time use............................................... 110
VI.4 Estimated FS initiative impacts: Health and safety ................................................................. 111
VI.5 Estimated FS initiative impacts: Business and adult time use................................................ 113
VI.6 Estimated FS initiative impacts: Economic well-being ............................................................ 115
VI.7 Estimated FS initiative impacts: Composition and mobility .................................................... 116
VII.1 Estimated impacts of connection on energy use .................................................................... 127
VII.2 Estimated impacts of connection on education and child time use ........................................ 129
VII.3 Estimated impacts of connection on health and safety ........................................................... 130
VII.4 Estimated impacts of connection on business and adult time use ......................................... 132
VII.5 Estimated impacts of connection on economic well-being ..................................................... 134
VII.6 Impact of connection: Matched comparison analysis versus instrumental variable
analysis ................................................................................................................................... 138
VII.7 Design element combinations analyzed ................................................................................. 141
VII.8 Composition of comparison group by design element ............................................................ 143
VII.9 Average bias across 59 outcome variables ............................................................................ 147
A.1 Description of weights .............................................................................................................. A.3
A.2 Variables included in the propensity score models ................................................................. A.7
A.3 Propensity score diagnostic statistics: T&D lines model ....................................................... A.11
A.4 Propensity score diagnostic statistics: Exploratory model ..................................................... A.14
B.1 T&D lines impact analysis: Baseline equivalence of community characteristics ..................... B.3
B.2 T&D lines impact analysis: Baseline equivalence of household characteristics...................... B.6
B.3 FS impact analysis: Baseline equivalence of household characteristics ............................... B.11
B.4 Exploratory impact analysis of connection to the grid: Baseline equivalence of
household characteristics ...................................................................................................... B.16
C.1 Full descriptions of community outcomes ................................................................................C.4
C.2 Full descriptions of household outcomes.................................................................................C.8
C.3 Full descriptions of control variables ......................................................................................C.14
C.4 Outcomes without a lagged (baseline) measure ...................................................................C.16
D.1 Energy and pollution produced per unit of different energy sources .......................................D.3

viii
TABLES MATHEMATICA POLICY RESEARCH
D.2 Energy-consuming devices: Energy use, pollution, and output estimates ..............................D.5
D.3 Conversion factors for water pumps ........................................................................................D.9
E.1 Community-level T&D lines impacts ........................................................................................ E.4
E.2a Household-level T&D lines impacts: Full analysis sample ...................................................... E.7
E.2b Household-level T&D lines impacts: Subgroup results by gender of household head .......... E.11
E.2c Household-level T&D lines impacts: Subgroup results by age of household head ............... E.12
E.2d Household-level T&D lines impacts: Subgroup results by urban-rural location..................... E.13
E.2e Household-level T&D lines impacts: Subgroup results by baseline income quartile ............. E.14
E.3a FS impacts: Full analysis sample .......................................................................................... E.15
E.3b FS impacts: Subgroup results by gender of household head ................................................ E.19
E.3c FS impacts: Subgroup results by age of household head ..................................................... E.20
E.3d FS impacts: Subgroup results by urban-rural location ........................................................... E.21
E.3e FS impacts: Subgroup results by baseline income quartile ................................................... E.22
E.3f FS impacts: Kigoma versus non-Kigoma regions .................................................................. E.23
E.4a Impacts of connection to grid electricity: Full analysis sample .............................................. E.24
E.4b Impacts of connection to grid electricity: Subgroup results by gender of household
head ....................................................................................................................................... E.27
E.4c Impacts of connection to grid electricity: Subgroup results by age of household head ......... E.28
E.4d Impacts of connection to grid electricity: Subgroup results by urban-rural location .............. E.29
E.4e Impacts of connection to grid electricity: Subgroup results by baseline income quartile ...... E.30
F.1 Examples of coordinates and calculation by type of observation ............................................ F.5
F.2 Percentage of household baseline locations within given thresholds of follow-up
locations before and after Solution 4 (selected illustrative communities; nonmoving
households only) ...................................................................................................................... F.7
G.1 Design element combinations analyzed ................................................................................. G.3
G.2 Outcomes used in within-study comparison ........................................................................... G.4
G.3 Composition of comparison group by design element ............................................................ G.6
G.4 Distances between control and comparison communities...................................................... G.7
G.5 Control communities by number of comparison communities within 30 kilometers ............... G.7
G.6 Final Propensity Score Model – Odds Ratio ......................................................................... G.12
G.7 Propensity scores for unmatched and matched samples using final propensity scores
based on rich covariates ....................................................................................................... G.13
G.8 Balance tests for variables in final propensity score model .................................................. G.14

ix
TABLES MATHEMATICA POLICY RESEARCH
G.9 Propensity scores for local comparison group and local matching ...................................... G.16
G.10 Statistical significance of bias reduction differences across design element
combinations ......................................................................................................................... G.18
G.11 Average bias by outcome domain ........................................................................................ G.19
G.12 Average bias across 59 outcome variables .......................................................................... G.21
G.13 Average bias by outcome domain ........................................................................................ G.22
H.1 First-stage regression results: Full sample ............................................................................H.12
H.2 First-stage regression results: Nonmovers only ....................................................................H.12
H.3 Impact of the instrumental variables on monthly electricity use at baseline: Full sample .....H.13
H.4 Impact of the instrumental variables on monthly electricity use at baseline:
NonmoverH. sample ..............................................................................................................H.14
H.5 Impact of connection on monthly electricity use: Full sample................................................H.15
H.6 Impact of connection on monthly electricity use: Nonmover sample .....................................H.15
H.7 Impact of connection on monthly electricity use: Nonmover sample without
households within 30-meter distance from nearest electric pole ...........................................H.16
H.8 Impact of connection on selected outcomes for robustness check: Full sample ..................H.17
J.1 T&D impacts on employment outcomes .................................................................................. J.4
J.2 FS impacts on employment outcomes ..................................................................................... J.4
J.3 Estimated impacts of actually connecting on employment ...................................................... J.5
K.1 Stakeholder comments ............................................................................................................ K.3

x