Accreditation, Certification, and Credentialing: Relevant Concerns for U.S. Evaluators PDF
Preview Accreditation, Certification, and Credentialing: Relevant Concerns for U.S. Evaluators
Number145 Spring2015 NewDirectionsforEvaluation PaulR.Brandon Editor-in-Chief Accreditation, Certification, and Credentialing: Relevant Concerns for U.S. Evaluators James W. Altschuld Molly Engle Editors ACCREDITATION,CERTIFICATION,ANDCREDENTIALING:RELEVANTCONCERNS FORU.S.EVALUATORS JamesW.Altschuld,MollyEngle(eds.) NewDirectionsforEvaluation,no.145 PaulR.Brandon,Editor-in-Chief Copyright© 2015 Wiley Periodicals, Inc., A Wiley Company, and the American Evaluation Association. All rights reserved. No part of this publication may be reproduced in any form or by any means, except as permitted under sections 107 and 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher or authorization through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923; (978) 750-8400; fax (978) 646-8600. The copyright notice appearing at the bottom of the first pageofachapterinthisjournalindicatesthecopyrightholder’sconsent that copies may be made for personal or internal use, or for personal orinternaluseofspecificclients,ontheconditionthatthecopierpay forcopyingbeyondthatpermittedbylaw.Thisconsentdoesnotextend tootherkindsofcopying,suchascopyingforgeneraldistribution,for advertisingorpromotionalpurposes,forcreatingcollectiveworks,orfor resale.Suchpermissionrequestsandotherpermissioninquiriesshould be addressed to the Permissions Department, c/o John Wiley © Sons, Inc.,111RiverStreet,Hoboken,NJ07030;(201)748-6011,fax(201) 748-6008,www.wiley.com/go/permissions. Microfilmcopiesofissuesandarticlesareavailablein16mmand35mm, aswellasmicrofichein105mm,throughUniversityMicrofilmsInc.,300 NorthZeebRoad,AnnArbor,MI48106-1346. NewDirectionsforEvaluationisindexedinAcademicSearchAlumniEdi- tion(EBSCOPublishing),EducationResearchComplete(EBSCOPub- lishing),HigherEducationAbstracts(ClaremontGraduateUniversity), SCOPUS (Elsevier), Social Services Abstracts (ProQuest), Sociological Abstracts(ProQuest),WorldwidePoliticalScienceAbstracts(ProQuest). NEWDIRECTIONSFOREVALUATION(ISSN1097-6736,electronicISSN1534- 875X)ispartofTheJossey-BassEducationSeriesandispublishedquar- terlybyWileySubscriptionServices,Inc.,AWileyCompany,atJossey- Bass, One Montgomery Street, Suite 1200, San Francisco, CA 94104- 4594. SUBSCRIPTIONSforindividualscost$89forU.S./Canada/Mexico/interna- tional. For institutions, $358 U.S.; $398 Canada/Mexico; $432 inter- national. Electronic only: $89 for individuals all regions; $358 for institutionsallregions.Printandelectronic:$98forindividualsinthe U.S.,Canada,andMexico;$122forindividualsfortherestoftheworld; $430 for institutions in the U.S.; $470 for institutions in Canada and Mexico;$504forinstitutionsfortherestoftheworld. Allissuesareproposedbyguesteditors.Forproposalsubmissionguide- lines, go to http://www.eval.org/p/cm/ld/fid=48. Editorial correspon- dence should be addressed to the Editor-in-Chief, Paul R. Brandon, UniversityofHawai‘iatMa¯noa,1776UniversityAvenue,CastleMemorial HallRm118,Honolulu,HI96822-2463. www.josseybass.com Coverphotographby©iStock.com/Smithore NEW DIRECTIONS FOR EVALUATION SponsoredbytheAmericanEvaluationAssociation EDITOR-IN-CHIEF PaulR.Brandon UniversityofHawai‘iatMa¯noa ASSOCIATEEDITORS J.BradleyCousins UniversityofOttawa Lois-ellinDatta DattaAnalysis EDITORIALADVISORYBOARD AnnaAhSam UniversityofHawai‘iatMa¯noa MichaelBamberger Independentconsultant GailBarrington BarringtonResearchGroup,Inc. FredCarden InternationalDevelopmentResearchCentre ThomasChapel CentersforDiseaseControlandPrevention LeslieCooksy SierraHealthFoundation FionaCram KatoaLtd. PeterDahler-Larsen UniversityofSouthernDenmark E.JaneDavidson RealEvaluationLtd. StewartDonaldson ClaremontGraduateUniversity JodyFitzpatrick UniversityofColoradoDenver JenniferGreene UniversityofIllinoisatUrbana-Champaign MelvinHall NorthernArizonaUniversity GeorgeM.Harrison UniversityofHawai‘iatMa¯noa GaryHenry VanderbiltUniversity RodneyHopson GeorgeMasonUniversity GeorgeJulnes UniversityofBaltimore JeanKing UniversityofMinnesota SavilleKushner UniversityofAuckland RobertLahey RELSolutionsInc. MiriLevin-Rozalis BenGurionUniversityoftheNegevandDavidsonInstituteat theWeizmannInstituteofScience LauraLeviton RobertWoodJohnsonFoundation MelvinMark PennsylvaniaStateUniversity SandraMathison UniversityofBritishColumbia RobinLinMiller MichiganStateUniversity MichaelMorris UniversityofNewHaven DebraRog WestatandtheRockvilleInstitute PatriciaRogers RoyalMelbourneInstituteofTechnology MaryAnnScheirer ScheirerConsulting RobertSchwarz UniversityofToronto LynShulha Queen’sUniversity NickL.Smith SyracuseUniversity SanjeevSridharan UniversityofToronto MonicaStitt-Bergh UniversityofHawai‘iatMa¯noa Editorial Policy and Procedures NewDirectionsforEvaluation,aquarterlysourcebook,isanofficialpublicationofthe AmericanEvaluationAssociation.Thejournalpublishesworksonallaspectsofevalua- tion,withanemphasisonpresentingtimelyandthoughtfulreflectionsonleading-edge issues of evaluation theory, practice, methods, the profession, and the organizational, cultural,andsocietalcontextwithinwhichevaluationoccurs.Eachissueofthejournal isdevotedtoasingletopic,withcontributionssolicited,organized,reviewed,andedited byoneormoreguesteditors. Theeditor-in-chiefisseekingproposalsforjournalissuesfromaroundtheglobeabout topics new to the journal (although topics discussed in the past can be revisited). A diversityofperspectivesandcreativebridgesbetweenevaluationandotherdisciplines, aswellaschaptersreportingoriginalempiricalresearchonevaluation,areencouraged. Awiderangeoftopicsandsubstantivedomainsisappropriateforpublication,including evaluativeendeavorsotherthanprogramevaluation;however,theproposedtopicmust beofinteresttoabroadevaluationaudience. Journalissuesmaytakeanyofseveralforms.Typicallytheyarepresentedasaseriesof relatedchapters,buttheymightalsobepresentedasadebate;anaccount,withcritique andcommentary,ofanexemplaryevaluation;afeature-lengtharticlefollowedbybrief criticalcommentaries;orperhapsanotherformproposedbyguesteditors. Submitted proposals must follow the format found via the Association’s website at http://www.eval.org/Publications/NDE.asp.Proposalsaresenttomembersofthejour- nal’sEditorialAdvisoryBoardandtorelevantsubstantiveexpertsforsingle-blindpeer review.Theprocessmayresultinacceptance,arecommendationtoreviseandresubmit, orrejection.Thejournaldoesnotconsiderorpublishunsolicitedsinglemanuscripts. Beforesubmittingproposals,allpartiesareaskedtocontacttheeditor-in-chief,whois committedtoworkingconstructivelywithpotentialguesteditorstohelpthemdevelop acceptableproposals.Foradditionalinformationaboutthejournal,seethe“Statement oftheEditor-in-Chief”intheSpring2013issue(No.137). PaulR.Brandon,Editor-in-Chief UniversityofHawai‘iatMa¯noa CollegeofEducation 1776UniversityAvenue CastleMemorialHall,Rm.118 Honolulu,HI968222463 e-mail:[email protected] C ONTENTS EDITORS’NOTES 1 JamesW.Altschuld,MollyEngle 1. TheInexorableHistoricalPressoftheDeveloping 5 EvaluationProfession JamesW.Altschuld,MollyEngle Theauthorsprovideanoverviewofthedevelopmentofthefieldof evaluationduringthepast40–50yearswithimplicationspertinentto accreditation,certification,andcredentialing. 2. CompetenciesforProgramEvaluatorsinLightofAdaptive 21 Action:What?SoWhat?NowWhat? JeanA.King,LaurieStevahn Thischaptercontainsadescriptionofkeytermsrelatedto competencies,currentworkgoingoninstudyingthem,issuesfor consideration,anduseoftheskillsintheevaluationfield. 3. TheStateofPreparingEvaluators 39 JohnM.LaVelle,StewartI.Donaldson Inthischapterareareviewoftrendsinevaluationeducation programs,recentadvancesintheprofessionalpreparationof evaluators,andadiscussionoffutureresearchonkeytopicsin evaluationeducation. 4. HowDoesAccreditationFitIntothePicture? 53 JamesC.McDavid,IreneHuse Usingbusinessmanagement,accounting,andhumanresource developmentasexamples,theaccreditationofinstitutionsor programsofferingevaluationeducationisthoughtfullyexamined. 5. CredentialedEvaluatorDesignationProgram,theCanadian 71 Experience KeikoKuji-Shikatani TherationalefortheCanadiansystemandhowitcameintobeing (theforcesthatshapedit,whatitlookslike,howitfunctions,issues, andotheraspects)arethemaintopicscovered. 6. EvaluatorCertificationandCredentialingRevisited:A 87 SurveyofAmericanEvaluationAssociationMembersinthe UnitedStates MichelleBaronSeidling ThischapteranalyzestheresultsofasurveyofAEAmemberswithin theUnitedStatesregardingthefeasibility,need,andpotentialfor certificationorcredentialinginthefuture. 7. Accreditation,Certification,Credentialing:DoesItHelp? 103 GeneShackman Thischapterisasynthesisoftheprioroneswithacarefullookatthe evidenceaboutwhetheraccreditation,certification,orcredentialing improvestheevaluationofprogramsorprojects. INDEX 115 E ’ N DITORS OTES A s charter members of the American Evaluation Association (AEA) and being active in its predecessor organizations, we have an ex- tensive history of discussing, reviewing, debating, and wondering about accreditation, certification, and credentialing (ACC). We are long- term collaborators (over 34 years) who have been involved continuously in presenting and writing about the preparation of evaluators and cer- tification. In the early 1990s as an AEA board member, Engle recruited (cajoled, actually) Altschuld to lead a new directory of evaluation prepa- rationprograms. This effort led to a New Directions in Program Evaluation (now New DirectionsforEvaluation[NDE])issue(No.62)publishedin1994.Wenote this earlier result because it relates to the current issue. The first coedi- tor was communicating with the NDE Editor-in-Chief about an issue on needsassessment(Engleisacontributortoit).Inthatconversation,itwas mentioned that she and I were recommended for an update to the 1994 publicationfocusingontheinterconnectedideasofACC. Wow! What an opportunity to reconsider a topic that was done two decades ago, especially given the vastly changing context of the practice ofevaluation,thetheoreticalunderstandingsofit,andwhathashappened andishappeningwithprofessionalorganizations.Weweredelightedtobe thoughtofforthetaskandtherestofthestoryis,astheysay,“history.” Thisissueisnotanupdateofthe1994effort,norisitintendedtobe:its focusisoninterconnectinganddissectingideasrelativetoACC.Wewanted to take a fresh look at them and we wanted our chapter authors to do the samething.Tothatend,weassembledadiversegroupofindividualswith uniqueviewpointstobeourwriters. Wedebatedhowtostartthecurrentjourneyandfeltthatlookinginto ACCnecessitatedplacingthethreecomponentsinthehistoricalcontextof how evaluation has evolved in the United States since the 1960s. We are notadvocatingforACC;rather,weilluminatewhattheirnatureisandhow past and current conditions have influenced thinking about them. We see the three parts of ACC as interrelated, and as critically important to any discussionofourfield. We start the discussion of the historical context by creating a time- linefromthe1960stothepresent(althoughwearesurethatsomeevents are omitted, making this a personal expression of history, not an actual one). This is the core content in Chapter 1 (“The Inexorable Historical Press of the Developing Evaluation Profession”) written by us. In what waysdothepastandthepresentinformandguideusrelativetoACC?The NAsEsWocDiaIRtiEoCnT.IONSPFuObRlisEhVeAdLUoAnTlIiOnNe,innoW.1il4e5y,OSpnrliinnge2L0ib1r5ary©(wi2l0ey1o5nWlinileelyibPrearriyo.cdoicma)ls,•InDc.O,aI:n1d0t.h1e00A2m/eevr.2ic0a1n0E7valuation 1 2 ACCREDITATION,CERTIFICATION,ANDCREDENTIALING chapter also contains definitions of terms and how changes in evaluation haveaffectedACC. Having set the stage, we challenged our authors to go beyond simply providinganupdateofwhatisgoingonatpresent;weaskedthemtodelve intothesubtleandproblematicconcernsthatarepartofallthecomponents of ACC. They were up to the responsibility, and we are grateful for their opennessandwillingnesstoacceptthischargewithoutflinching. The focus of Chapter 2 (“Competencies for Program Evaluators in LightofAdaptiveAction:What?SoWhat?NowWhat?”),byJeanA.King andLaurieStevahn,addressesthecompetenciesthatdefineevaluators.This is a tricky consideration, and an ACC issue without it would certainly miss the mark. These two authors, along with others, have been study- ing competencies (and competent practice) for nearly 15 years and are known throughout the world for their publications and in-depth research and thinking about this area. They have done an excellent job of dealing withconcepts,examiningsalientresearch,andthenraisingquestionsthat force evaluators to be more thoughtful in regard to specifying what they bringtothetableviatheirservicestoprogramsandprojects. Withthisbackdropofcompetencies,itwouldseemasimplematterto describe the educational environment necessary for evaluators. Put sim- ply, “Where do individuals acquire the skills and culture of the profes- sionalevaluator?”Weknewtheanswerwasn’tstraightforwardandsought out John M. LaVelle and Stewart I. Donaldson for preparing Chapter 3 (“TheStateofPreparingEvaluators”).Inthepastfewyearstheyhavebeen conductingresearchaboutthenatureofprogramsatuniversitiesandjustre- centlycompletedanotherstudy.Butlikeeverythingelseinthefield,prepa- ration is now extensively offered in many venues other than academia. What does the educational picture look like and what might it become in the near future? The landscape they draw for us opens vistas to a quite complexscene. What role does or should accreditation play in the ACC scenario was the question posed to James C. McDavid and Irene Huse for Chapter 4 (“How Does Accreditation Fit Into the Picture?”). McDavid and Huse are Canadianswhoauthoredamajortextonevaluationandwhowereinstru- mental in writing the 2006 literature review that was the underpinning of the2009CanadianEvaluationSociety’sCredentialedEvaluatorDesignation system for program evaluators. With expanding educational and employ- ment options, the field must be concerned with accreditation of evaluator preparation programs. How should evaluation as a budding profession go aboutensuringtheirvalidityandquality?Thisconsiderationwasraisedby AEA in the 1990s and is as meaningful now as it was then. McDavid and Huse’svastknowledgeofthefieldgetsusclosertoaresponse. Asbrieflymentionedabove,Canadainitiatedasystemthatisbecoming known to many evaluators in the United States and in the world. It could NEWDIRECTIONSFOREVALUATION•DOI:10.1002/ev
The list of books you might like
