Table Of Content

future internet Article ODK Scan: Digitizing Data Collection and Impacting Data Management Processes in Pakistan’s Tuberculosis Control Program SyedMustafaAli1,*,RachelPowers2,JeffreyBeorse3,*,ArifNoor1,FarahNaureen1,*, NaveedAnjum1,MuhammadIshaq1,JavariyaAamir1andRichardAnderson3 1 mHealthCoordinator,MercyCorps,PakPalace,MurreeRoad,RawalChowk,Islamabad44000,Pakistan; [email protected](A.N.);[email protected](N.A.);[email protected](M.I.); [email protected](J.A.) 2 InformationSystems,VillageReach,Seattle,WC98102,USA;[email protected] 3 ComputerScienceandEngineering,UniversityofWashington,Seattle,WA98101,USA; [email protected] * Correspondence:[email protected][email protected](S.M.A.); [email protected](J.B.);[email protected](F.N.);Tel.:+92-30-1855-0454(S.M.A.); +1-206-221-7701(J.B.);+92-30-2850-6178(F.N.) AcademicEditor:DinoGiuli Received:30August2016;Accepted:12October2016;Published:24October2016 Abstract: Thepresentgrievoustuberculosissituationcanbeimprovedbyefficientcasemanagement and timely follow-up evaluations. With the advent of digital technology, this can be achieved throughquicksummarizationofthepatient-centricdata. Theaimofourstudywastoassessthe effectiveness of the ODK Scan paper-to-digital system during a testing period of three months. Asequential,explanatorymixed-methodresearchapproachwasemployedtoelucidatetechnology use. Training,smartphones,theapplicationand3G-enabledSIMswereprovidedtothefourfield workers. Atthebeginning,baselinemeasuresofthedatamanagementaspectswererecordedand comparedwithendlinemeasurestodeterminetheimpactofODKScan. Additionally,attheendof thestudy,users’feedbackwascollectedregardingappusability,userinterfacedesignandworkflow changes. Atotalof122patients’recordswereretrievedfromtheserverandanalysedintermsof quality. ItwasfoundthatODKScanrecognized99.2%ofmultiplechoicefill-inbubbleresponsesand 79.4%ofnumericaldigitresponsescorrectly. However,theoverallqualityofthedigitaldatawas decreasedincomparisontomanuallyentereddata. UsingODKScan,asignificanttimereduction isobservedindataaggregationanddatatransferactivities,butdataverificationandform-filling activities took more time. Interviews revealed that field workers saw value in using ODK Scan, buttheyweremoreconcernedaboutthetime-consumingaspectsoftheuseofODKScan. Therefore, itisconcludedthatminimaldisturbanceintheexistingworkflow,continuousfeedbackandvalue additionsaretheimportantconsiderationsfortheimplementingorganizationtoensuretechnology adoptionandworkflowimprovements. Keywords: mHealth; ODK scan; mobile health application; digitizing data collection; data management processes; paper-to-digital system; technology-assisted data management; treatmentadherence 1. Introduction Tuberculosis(TB)continuestobeaglobalandgravepublichealthconcern. Itisahealthissue formillionsandthecauseofnumerousdeathseachyearworldwide[1]. Despitetheavailabilityof effectivetreatmentsandstrategiestoimproveaccesstotreatmentregimens,multidrug-resistantand FutureInternet2016,8,51;doi:10.3390/fi8040051 www.mdpi.com/journal/futureinternet FutureInternet2016,8,51 2of17 extremelydrug-resistantTBcasesareemergingduetothelackofadherencetotreatmentprotocols. Thisnon-adherenceismainlybecauseofthelongertreatmentcourse(6to8months)andunpleasant sideeffectsofthedrugtreatment[2]. PakistanisaTBhigh-burdencountry,experiencingtheemergenceofalargenumberofnewTB cases(approximately420,000everyyear)andprevalenceofmultidrug-resistantTB.Pakistanranks fifthfordrug-susceptibleTBandfourthformultidrug-resistantTB[3]. Theburdenofdiseasecaused byTBandtheeffectivenessofdifferentprogrammaticinterventionsarebetterunderstoodthrough an interpretation of the available epidemiological data [4]. The program management function, Monitoring and Evaluation (M&E), makes the best use of the available resources (including data) to guide and support programmatic decision-making. The M&E system plans and conducts data collectionandanalysistoinformprogrammanagersandkeepperformancemeasuresontrack[5]. Unfortunately,thisimmenseresourceiseitherunderutilizedornotusedatall[4]. Health information systems used in disease-specific health programs are designed to collect patient-centricdatatogenerateactionableinformationtoguidedecision-making. Likewise,TBcontrol programsmostlymaintainandmanagepaper-basedpatientrecordingandreportingsystems[6]. Fidelity to treatment protocols is ensured through case management (DOTS strategy) and follow-upevaluations(sputumspecimenformicroscopicexaminationorcultureatspecificpointsin thetreatmentperiod)[7]. Generally,intheTBcontrolprograms,paper-basedpatientrecordsareused toprepareperiodicreportsonthepatientenrolment,retentionandtreatmentprogress. Thesedata summariesarealsosenttodifferentmanagementlevelsforaggregation,analysisandreporting[6]. However, paper-based recording and reporting systems suffer from some inherent challenges, whichinclude:checkingdataquality,dataentryandaggregation,allottingtreatmentoutcomecorrectly, fillinggapsindata,analysingdataandgeneratingactionableinformation. Thesedatamanagement activities are inefficient, work-intensive and time consuming [8]. However, the advent of digital technologyhaspositivelyimpactedtheorganizationalworkflow[9]. ElectronicHealthInformationSystems(eHIS)havegeneratedbetterresultsonthethreefrontsof TBcontrolprograms: patientcare,resourcemanagementanddiseasesurveillance[8]. Increasingly, mobilehealth(mHealth)interventionsaregrabbingtheattentionofthepublichealthprofessionals becauseoftheirabilitytoimprovethepublichealthsituation[10]. mHealthreferstotheapplication of portable hand-held devices and communication capabilities in the health care programs [11]. Duringthepastdecade,mHealthhasbeencontinuouslyevolvingwithintheeHealthdomain,whereit hasshowngreatpotentialtoovercomemanytraditionalbarriersindeliveringcareandreducingthe associateddistance,timeandcost[12]. Fortunately,mostdevelopingcountrieshavealreadylaunched mHealthinitiativestounleashthepowerandreachofsmartphonesandconsiderthemanimportant resourceforfrontlineworkers[10]. Early diagnosis and vigilant management are important considerations to successfully control TB. Therefore, in TB control programs, case management involves a multidisciplinary approach for comprehensive and coordinated follow up of the presumptive and confirmed TB case[13]. Thisapproachincludesimprovedcommunicationbetweenpatientandcaregivingteams, patienteducationanddeliveryofsupportduringtreatment[14]. Increasingly,mHealthapplications arebeingusedtosupporthealthcareteamsinearlydiagnosisandmonitoringpatientprogress[15]. ManyoftheexistingmHealthsolutionsforTBcasemanagementarecustomsoftwaredevelopedfor individual organizations and use cases, they are not freely available for other organizations to try. TherearealsogeneralpurposemHealthsolutionsavailable,suchasMedicMobileorODKCollect anditscousins. Thesearedirect-to-digitalsystemsthatdonotinvolveanypaperrecordkeepingor paperarchives. Among the various reported uses of mHealth tools the most reported are: data collection, electronicdecisionsupport,provider’seducationandtraining,andcommunication[10]. However, theparticularuseofmHealthfordatacollectionhasagreaterlikelihoodofimprovingpublichealth in low- and middle-income countries [16]. Noteworthy are those unpredictable implementation FutureInternet2016,8,51 3of17 challengesandcomplicatedsituationalfactorsthataffectanintroductionofmHealthapplicationsin primaryhealthcaresettingsofthedevelopingworld[17,18]. Sincethereareanumberoffreely-availableopensourcemHealthapplicationsaimedatimproving datacollection,onlyonetoolleveragedthestrengthsoftheexistingpapersystemwiththeflexibility and efficiency of the digital records. We chose the ODK Scan paper-to-digital application to test improved data collection in the public–private mix model of the TB control program of Pakistan. TheultimategoalofthisinterventionwastoreducethetimeandresourcesspentonTBdatacollection byimprovingthedatacollectionanddigitizationprocessandreducingrelianceonmanualdataentry. Additionally,ODKScanwaschosenbecauseitallowssupervisorsandmanagerstoassessthedata qualitybyreviewingtheoriginalimagesnippets. Thespecificobjectivesofthispilotstudywereto: (1)assessdataqualityaspectsatboththeappanduserlevels,(2)assessthequantitativeimpactof ODKScanonthedatamanagementprocesses,and(3)collectfeedbackonapplicationuserexperiences toascertainappusability,userinterfacedesignandworkflowchanges. 2. MaterialandMethods 2.1. DescriptionofTechnology OpenDataKit(ODK)ScanisanAndroid-basedmobileapplicationandapartoftheODK2.0 suiteoftools[9,19]. Itusesthedevice’sbuilt-incameratocaptureanimageofascan-compatiblepaper form, which it feeds into a series of image processing algorithms (SURF, fast approximate nearest neighbor,andRANSAC)toestablishapoint-by-pointmappingbetweenthetemplateandthecaptured image [20–23]. Using this mapping, Scan segments the form into small image snippets that each containsasingledatafieldfromtheoriginalform[24]. Theimagesnippetsfallintothreepossible prompttypes: hand-filledbubbles,structurednumberboxes,andQRcodes. Hand-filledbubblesare multiple-choicequestionsthatarecompletedbyfillinginthebubblethatcorrespondswiththedesired response. Scanclassifieshand-filledbubblesusingaSupportVectorMachine(SVM)andprincipal componentanalysis(PCA)[20,25,26]. Structurednumberboxesarea2×3gridofdotsthatcanbe connectedbystraightlinestodrawnumerals. ScanclassifiesnumbersusingaMulti-layerPerceptron (MLP)withbackpropagationoferror[27]. Finally,QRcodesaretwo-dimensionalbarcodesthatcan beprocessedasalphanumericstrings. ScanusestheZebraCrossing(Zxing)librarytoclassifyQR Codes[28]. AfterScanhasclassifiedeachprompt,itinsertsthedataintoaneditablerecordintheODK 2.0database.Eachdigitizedfieldincludesacopyofitsassociatedimagesnippetfromtheoriginalform, whichcanbeusedtoverifydigitizedfields. OncetherecordisintheODK2.0database,userscanuse theothertools,suchasODKSurveyandODKTablestoview,edit,verify,aggregate,andsyncthedata orcreatecustomizedservices,interactionsandreportswiththedata[19]. ODKScandoesnotprocessfreehandwrittentext. Promptsthatrequirewrittentextwillinclude boxes on the paper forms that users can fill with their hand written text, but Scan will leave the correspondingfieldinthedatabaserecordblank. Handwrittentextfieldscanbemanuallytranscribed onthedevicebyviewingthepromptinODKSurvey,whichwilldisplaythecorrespondingimage snippet, and typing in the text via the phone’s keyboard. Alternatively, they can be manually transcribedonaPCaftertherecordhasbeensynced,againbyviewingtheimagesnippet. Scan and the rest of the ODK 2.0 tools do not require a consistent Internet connection to function[19]. Alloftheimageprocessingisperformedbythedeviceitself,andthedataisstoredina localdatabase. AnInternetconnectionisonlyrequiredforsyncingrecordswiththecloud,andthis canbedoneattheleisureoftheuser. 2.2. ODKScanWorkflow BeforeODKScanistobeused,ascan-compatibleformmustbedesignedusingtheformdesigner. Paper copies of the form image are printed out and the form’s definition files are copied to the Android-baseddevice. Aftertheformisfilled,theusertakesapictureusingthebuilt-incameraof FutureInternet2016,8,51 4of17 Future Internet 2016, 8, 51   4 of 16  of the smartphone. ODK Scan detects the data fields and auto‐digitizes them. The user can then  thesmartphone. ODKScandetectsthedatafieldsandauto-digitizesthem. Theusercanthenlaunch launch ODK Survey to view and verify data fields, and editing if necessary. After a user is sure about  ODKSurveytoviewandverifydatafields,andeditingifnecessary. Afterauserissureaboutthedata the data quality, the patient record is synced with an aggregate server. If no internet connection is  quality,thepatientrecordissyncedwithanaggregateserver. Ifnointernetconnectionisavailable,the available, the user can delay synchronization and continue collecting data until connectivity is  usercandelaysynchronizationandcontinuecollectingdatauntilconnectivityisrestored. Atanytime, restored. At any time, the user can view all synced records on the smartphone and run customized  theusercanviewallsyncedrecordsonthesmartphoneandruncustomizedreportstoviewactionable reports to view actionable information. For this trial we created a report that generates information  information. Forthistrialwecreatedareportthatgeneratesinformationaboutpatientswhosefollow about patients whose follow up appointments are either overdue or due in the coming days. Aside  upappointmentsareeitheroverdueordueinthecomingdays. Asidefromthesyncingstepofthe from the syncing step of the ODK Scan workflow, all steps can be undertaken without an internet  ODKScanworkflow,allstepscanbeundertakenwithoutaninternetconnection. connection.   2.3. Intervention 4. Intervention   AmongninerecordingandreportingformatsusedintheTBcontrolprogramofPakistan,one Among nine recording and reporting formats used in the TB control program of Pakistan, one  recordingformat(TB03)wasselected(Figure1). Itsscan-compatibleformwasdesignedusingthe recording format (TB03) was selected (Figure 1). Its scan‐compatible form was designed using the  ODK Scan Form Designer to create machine-readable fields for the app to recognize (Figure 2). ODK Scan Form Designer to create machine‐readable fields for the app to recognize (Figure 2). The  The scan-compatible form consists of three types of machine-readable data fields, i.e., structured scan‐compatible form consists of three types of machine‐readable data fields, i.e. structured number  numberboxes,fill-inbubblesandQRcodes,andonenon-machinereadabledatafield,i.e.,handwritten boxes, fill‐in bubbles and QR codes, and one non‐machine readable data field, i.e. hand written text  textfields(Figure3). Aftertheformdesignwasprepared,itwaspre-tested,necessarymodificationsin fields (Figure 3). After the form design was prepared, it was pre‐tested, necessary modifications in  formdesignweremade,andtheformswereprintedandboundintoaregisterbook. Theintervention form design were made, and the forms were printed and bound into a register book. The intervention  started at the beginning of May 2016 and lasted for three months in district-level field settings. started at the beginning of May 2016 and lasted for three months in district‐level field settings.  Informed consent was sought from four of the field workers, belonging to two enrolled districts Informed consent was sought from four of the field workers, belonging to two enrolled districts  (ChakwalandHafizabad)oftheTBcontrolprogramofPakistan. Thefieldworkers(DistrictField (Chakwal and Hafizabad) of the TB control program of Pakistan. The field workers (District Field  SuSupperevrvisisoorrss)) wweerree ttrraaiinneedd iinn uusisningg thteh eScSacna‐nco-cmopmaptiabtlieb TleB T0B3 f0o3rmfo arnmd athned atphpe waporpkfwloowrk aflnodw waenred  wgeirveeng ivsemnasrmtpahrotnpehso n(Seasm(Ssaumngs uGnaglaGxayl aJx7y aJ7nda nHduHauwaewi eYi6Y) 6a)nadn dscsacna‐nc-ocmompaptaibtilbe leTTBB030 3rreeggisitsetersr.s . AAdddidtiiotinoanlalyll,yw, wepe rporvoivdieddedth tehmem3G 3-Ge‐neanbalbedledm moboibleilSeI SMIMsfso froirn itnetrenrentecto cnonnenceticvtiivtyitayn adndm monotnhtlhyltyo tpo-pu‐p fourps hfoarr isnhgadriantga vdiaatain vteiar ninett.erWneeti.m Wpeo simedpnooserde sntroi crteiostnriocntioancc oenss aincgceosrsidnogw onr ldoaodwinnlgoaandyinogt haenrym oothbeilre  apmpolbicialeti oanpsp.liFciaetlidonwso. rFkieelrds wweorrekaerllso wweedret oalulosweesdm atort puhseo nsmesa,rintpchluodniensg, imncalkuidnigngca mllsa,kteinxgt mcaelslss,a gteixntg  anmdesinsategrinnegt abnrdo winstienrgn.et browsing.     FFigiguurree1 1..O Orriiggiinnaall TTBB0033 ffoorrmm uusseedd iinn rroouuttiinnee ddaattaa mmaannaaggeemmeenntt pprroocceessss.. FutureInternet2016,8,51 5of17 Future Internet 2016, 8, 51   5 of 16  Future Internet 2016, 8, 51   5 of 16      Figure 2. Scan‐compatible TB03 form used in ODK Scan paper‐to‐digital system displaying fill‐in  Figure2. Scan-compatibleTB03formusedinODKScanpaper-to-digitalsystemdisplayingfill-in Fbiugbubrlee s2, .n Sucmanb‐ecro bmopxaetsi balned T tBex0t3 f ifeolrdms.  used in ODK Scan paper‐to‐digital system displaying fill‐in  bubbles,numberboxesandtextfields. bubbles, number boxes and text fields.  (a)  (b) (a)  (b)   (c)  (d)   (c)  (d) Figure 3. Formats of data fields on ODK Scan‐compatible TB03 form. (a) Machine‐readable structured  Fniugmurbee r3 . bFooxrems;a t(sb o) f mdaatcah fiineeld‐rse oand aObDleK f iSlcl‐ainn ‐cboumbbplaetsi;b l(ec )T Bu0s3e rf otrrman. s(car)i Mbeadc hteinxet ‐rfeiealdda; b(lde) s tmruaccthuirneed‐  Figure3.FormatsofdatafieldsonODKScan-compatibleTB03form.(a)Machine-readablestructured nreuamdabbelre  bQoRx ecso; d(eb. ) machine‐readable fill‐in bubbles; (c) user transcribed text field; (d) machine‐ numberboxes;(b)machine-readablefill-inbubbles;(c)usertranscribedtextfield;(d)machine-readable readable QR code.  QRcode. FutureInternet2016,8,51 6of17 2.4. Design A sequential, explanatory mixed-method research approach was employed to elucidate the technologyuseforaneffectiveimplementationoftheODKScanpaper-to-digitalsystem. However, themixedmethodresearchapproachinvolvescombiningthequantitativeanalysiswithqualitative reasoning. Thisresearchapproachincludesapplicationofthreemethods: (1)quantitativeassessment ofthedataqualityatbothappanduserlevels;(2)quantitativemeasurementofthetimespentinfilling forms,datatransfer,aggregationandverification;and(3)qualitativefeedbackfromthefieldworkers (users)tocapturetheircompleteexperience. However,eachofthesemethodsisexploredinmore detailbelow. (1) Quantitative assessment of the data quality was done at both the app and user levels. Attheapplication level, accuracy of ODK Scan’s ability to recognize fill-in bubbles and number boxeswasmeasuredbycomparingtherawScanvalues(thedigitizedvalueswhichScan’salgorithm producedbeforehumanverification)withthedataappearinginimagesnippetsofthecorresponding data fields. To calculate this, the team hired a Seattle USA-based data analyst to examine each individual number and fill-in bubble image snippet, to record the observed “ground truth” value as observed from the image, and to compare it with the corresponding raw value that ODK Scan processing recorded. Comparison of these field value pairs allowed us to calculate ODK Scan’s accuracyratesforallthenumberandfill-inbubblefields. At the user-level, the workflow allowed users to improve the quality of the data through verificationandeditingdatafieldsmanually. User-leveldataqualitywasassessedbycomparingthe same“groundtruth”valuesforeachfieldtothefinalvaluesthatthefieldworkersyncedwiththe server. Theseaccuracyrateswerepredictedtobehigherthantheapp-levelaccuracyratebecausethe fieldworkershadtheopportunitytocorrecterrorsfromthe“raw”Scanprocessing. Furthermore, we calculated the accuracy rates of the existing (non-ODK Scan) manual entry processesbycomparingthefieldvaluesobservedfromthehardcopyTB03registerpagestothevalues manuallyenteredbythedataentryworkerintheMicrosoftExceldatasheets. Thiswouldprovide us with a baseline for manual entry accuracy rate. Accuracy rates for both user-level digitization processes were calculated based on the full field value, rather than by individual digit or bubble (binaryassessmentofcorrectorincorrectforeachfield). (2)Quantitativemeasurementsofthetimespentinfillingforms,datatransfer,aggregationand verificationwererecordedintheroutinedatamanagementprocessusingdirectobservation.Theywere thencomparedwiththeendlinemeasuresforthesamedatamanagementprocessesusingtheODK Scansystem. (3) Gathering of qualitative feedback on app use experience was done through surveys and interviews. The survey captured Likert-scale assessment, while face-to-face interviews using a semi-structuredquestionnairewerecarriedoutwitheachofthefieldworkers. Theirexperienceswith appusabilityandworkflowmodificationwereexploredtoassesstheeffectivenessoftheODKScan paper-to-digitalsystemfromausabilityperspective. 3. Findings 3.1. DataQuality App-level quality measure: During an intervention period of three months, 122 patients’ paper-basedrecordswerescanned,digitizedandcheckedforquality.Allofthesedataweresyncedand availableonthecloud-basedODKaggregateserverforaccessandremoteanalysis.Thefirsttaskwasto assessODKScan’sabilitytocorrectlyparsetheuserfilledmachine-readabledatafields,thestructured numberboxesandthefill-inbubbles. Accuracyratesforthesefieldswerecalculatedbymanually reviewingthe122records,consistingof10,600digitsand8000fill-inbubbles,andcomparingScan’s rawpredictedvalueagainstthegroundtruthvalueoftheimagesnippet. Therecognitionaccuracyfor thefill-inbubbleswasfoundtobemuchhigherthanthatofthenumberboxes. Includingblankfields FutureInternet2016,8,51 7of17 (fieldsthatusersdidnotfillin),theScanappclassified99.2%fill-inbubblesand79.4%digitscorrectly (Table1). Table1.Measurementofaccuracyratesforrecognitionoffill-inbubblesandnumberboxes. Future Internet 2016, 8, 51   7 of 16  Table 1. MeasTuyrpemeoenftF oife aldccuracy rAatlelsF fioerl drescognitioBnl aonf kfillF‐iine lbdusbEblxecsl uandde dnumber boxes.  Number 79.4% 77.6% Fill-TinypBeu obfb lFeiseld  9A9l.2l %Fields Blank Fie9l7d.2s% Excluded  Number  79.4%  77.6%  Fill‐in Bubbles  99.2%  97.2%  The misclassified number fields were further investigated by mapping accuracy measures of thenumberboxesontheTBformtospecificfieldsonthephysicalpaper. Itwasfoundthatfields The misclassified number fields were further investigated by mapping accuracy measures of the  locatendumobnerth beoxreisg hont tshide eTBo ffotrhme top aspgeec,iftihc efiesliddse ofna trhteh epshtyfsricoaml pathpeerr. eItg wisatse rfobuonodk th’satb fiinelddisn lgo,castuedff ered fromolno wtheer riagchctu sriadcey ofm theea spuagrees, tthhea snidteh fearnthuemst bfreormb othxee srewgishtiecrh bowoekr’se blioncdaitnegd, sculfofesreerdt ofrotmhe locwenerte r or left-haacncdursaicdye m(eFaisguurrees t4h)a.nI nthsep neuctmiobnero bfotxhees wcahpicthu rwederei mloacagteesd schlooswere tdo tthhea tcetnhteerp oarg leesft‐whaenrde sbiedne ding away(Ffrigoumret h4)e. Ibnisnpdeicntigo.nT ohf itshbe ecnadptiunrgedw iamrpageeds tshheowpeodin tth-atot -tphoe ipnatgmesa wpperien gbeonfdtihnge caawpatyu frreodmi mthaeg eto binding. This bending warped the point‐to‐point mapping of the captured image to the template file,  thetemplatefile,whichcausedtheexpectedimagesnippetcoordinatestomismatch. Theresulting which caused the expected image snippet coordinates to mismatch. The resulting image snippets  imagesnippetsweremisaligned,meaningthattherewouldbemissingedgesandotherimportant were misaligned, meaning that there would be missing edges and other important information. The  information. The structured number classifier’s accuracy suffered when attempting to classify structured number classifier’s accuracy suffered when attempting to classify misaligned image  misalignedimagesnippets. snippets.    Figure 4. Mapping of the accuracy measures of the number boxes.  Figure4.Mappingoftheaccuracymeasuresofthenumberboxes. User‐level quality measure: The ODK Scan workflow includes viewing, editing and verifying  User-levelqualitymeasure: TheODKScanworkflowincludesviewing,editingandverifying data before it is synced with the server. Field workers were therefore given complete responsibility  databfoerf othree iqtuisalsityyn acsesdeswsmitehntth aensde rimveprr.oFvieemldenwt.o Hrkoewrsevwere,r eint htheer eefxoirsetinggiv menancuoaml dpaletate ernetsryp opnrosicbeislsi,t yfor thequthailsi rtyesapsosnessibsimliteyn its ashnadreidm bpertwoveeenm ae fnietl.dH woowrkeevr earn,din a tdhaetae mxiasntianggemmeannt oufafilcdera (tDaMenOt)r.y Thper oDcMesOs, this respoins sriebsipliotynsiisblseh aforre ddabteat weneteryn aanfide lqduwaliotyrk aetr tahned naexdta lteavmel aonfa mgeamnaegnetmoefnfitc. eWr(hDenM dOa)t.aT qhueaDlitMy Ois respo(nascicbulreacfyo)r odfa tsaynecnetdr yreacnodrdqsu waleirtey cahtetchkeedn eaxntdl ecvoemlopfarmeda nwaigthe mtheen tq.uWalihtye nofd amtaanquuaalllyit yen(taecrceud racy) records, it was found that the existing manual data entry process resulted in a data accuracy rate of  ofsyncedrecordswerecheckedandcomparedwiththequalityofmanuallyenteredrecords,itwas 94%, while the ODK Scan process only reached 86%. As described in the methodology section, both  foundthattheexistingmanualdataentryprocessresultedinadataaccuracyrateof94%,whilethe of these accuracy rates were calculated by comparing values observed from the paper records with  ODKScanprocessonlyreached86%. Asdescribedinthemethodologysection,bothoftheseaccuracy the values that were saved to the digital record in each process. While it is understandable that human  accuracy for data entry did not reach a perfect 100% for either processes, the discrepancy between FutureInternet2016,8,51 8of17 rateswerecalculatedbycomparingvaluesobservedfromthepaperrecordswiththevaluesthatwere savedtothedigitalrecordineachprocess. Whileitisunderstandablethathumanaccuracyfordata entrydidnotreachaperfect100%foreitherprocesses,thediscrepancybetweenthetworatesshows thattFhuetutrae sInkte-rdneetp 20e1n6d, 8e, n51c e  offield-workerfordataverificationwithODKScandidnotmeetth8 eof q16u ality oftheexistingmanualdataentry. ThisdiscrepancybetweenthemanualdataentryandverifiedODK the two rates shows that the task‐dependence of field‐worker for data verification with ODK Scan  Scandataaccuracyratesisfurtherexploredinthediscussionsection. did not meet the quality of the existing manual data entry. This discrepancy between the manual data  entry and verified ODK Scan data accuracy rates is further explored in the discussion section.   3.2. ImpactingDataManagementProcesses 7. Impacting Data Management Processes   In the TB control program, the routine data management process begins with the collection of data frIonm thael lTBen croonltlreodl pgreongerraaml, pthrae crtoituitoinnee rdsat(aG mPsa)n.agOenmceentt hperoccoensss obleigdiantse wditpha tpheer c-bolalescetdiorne opfo rt is prepadraetda iftroismt raalnl sefnerrorleledd togetnheeranl epxrtalcetivtieolnoefrsm (aGnPasg). eOmnecne tt(hseu bco-rnescoilpidieatnetdo pffiapcee)r,‐bwasheidc hretphoernt pisa sses throupgrhepmaruedlt iipt ilse tdraensskferrerevdie two sthaen ndexits lefivneal lolyf meannteargeedmeinntt o(suanb‐rEexcicpeilenstp orfefaicdes),h weehti.chT thheend pigasitsiezsa tion through multiple desk reviews and is finally entered into an Excel spreadsheet. The digitization of  ofpaper-basedrecordstakesasignificantamountoftimebeforeitissharedwiththenextlevelof paper‐based records takes a significant amount of time before it is shared with the next level of  management (principle recipient office) for finalization and progress review meetings (Figure 5). management (principle recipient office) for finalization and progress review meetings (Figure 5).  Timespentinform-filling, datacollection, transfer, aggregationandverificationwasmeasuredby Time spent in form‐filling, data collection, transfer, aggregation and verification was measured by  the direct observation. At baseline, these measures were recorded by observing the routine data the direct observation. At baseline, these measures were recorded by observing the routine data  managementprocess. However,attheendline,thesamemeasureswererecordedbyobservingthe management process. However, at the endline, the same measures were recorded by observing the  ODKOSDcaKn Swcaonr kwfloorkwfl.ow.     Figure 5. Routine data management process in the TB (Tuberculosis) control program of Pakistan.  Figure5.RoutinedatamanagementprocessintheTB(Tuberculosis)controlprogramofPakistan. Filling in the ODK scan‐compatible TB form took longer than the old form. Extra attention was  FillingintheODKscan-compatibleTBformtooklongerthantheoldform. Extraattentionwas required for filling in bubbles and writing structured numbers on the guiding dots correctly. Each  requifrieedldf owrofirklleinr g(ni n= b4u) bwbalse soabnsedrvwedri twinhgilest rfiullcintugr eind tnhureme bSecrasno‐cnomthpeagtiublied ifnogrmdso. tTshceo rarveecrtalyg.e Etaimche field worktearke(nn =to 4f)ilwl 1a2s Socbanse‐crovmedpawtibhliel efofirlmlisn gwains 0th2 rmeeinS 5c7a ns- c(momaxp, a0t3i:b04le sf; omrmins, .01T:h48e sa)v. eHroawgeevteimr, eont aken tofilla1v2erSacgaen it- ctooomkp 1a mtibinle anfodr m27s s wtoa fsil0l 2inm onine 5p7atsie(nmt raexc,o0rd3 :u04sinsg; mthien e,x0i1st:4in8gs T).BH03o rwegeivsteerr,. oTnhea nveexrta geit tooka1smpeicnt oafn ddat2a7 msatnoagfielmliennto, ndeatpa acotilelencttiroenc, owrdas unsoitn agffethcteede xbiys ttihneg OTDBK0 S3craeng wisotrekr.floTwh ebenceaxutsea sthpee ctof process remained the same: visit all of the enrolled GPs of a district. Finally, with the routine data  datamanagement,datacollection,wasnotaffectedbytheODKScanworkflowbecausetheprocess management process, once data collection was completed, it took 1–2 days to transfer data via local  remainedthesame: visitalloftheenrolledGPsofadistrict. Finally,withtheroutinedatamanagement courier service, in the form of hard‐copies. However, with the ODK Scan process the same transfer  process,oncedatacollectionwascompleted,ittook1–2daystotransferdatavialocalcourierservice, of data only lasted a few min. It only required the user to get to an area of internet connectivity and  intheformofhard-copies. However,withtheODKScanprocessthesametransferofdataonlylasted send the data to the ODK Aggregate Server. Future Internet 2016, 8, 51   9 of 16  FutureIFnutteurrnee Itn2te0r1n6e,t 820,1561, 8, 51   9 of 196 of17 In the routine data management process, aggregation of records required manual data entry into  In the routine data management process, aggregation of records required manual data entry into  aafFe uswtpurarmee  sIanipndtre.esrnahIetdet ose2hn0t,e1l e6ayt,n ,8r da,e n5q d1du i  igdriiegtdiiztiatzhtaietoiounn so eofrf o toonneeg  erreetcctooorrdadn  ttooaoorkek a2 2 mo mfiniin.n Tt.e hTrinsh edisti gcdioitginzinatteiizcoatnitv ipoirtnoy cpaersnosd cwesasessn  rwdedathus cereeddda tutoac 1et0do  9tt oho ef1 106   OFFsDFu uuiKtttnuuurr rsAeete   h IIiIgnnnnetgtt  eeetOrrrrhnneneDeeget tt O a K2220t00D e111S6K66Sc,,,    ea888S,r,n, c  5v55a 111wen   r    .owrokrfkloflwow. .M Moorreeoovveerr,,  tthhee  ddaatata v vereirfiicfaictiaotnio pnr opcreoscs ewssa sw garse agtlrye aatltleyr eadlt, earse din,  tahse  i99n9   oo otfff h  111e666     ODIKnOI DnSthc Kteah nSreco  prauonrtuo ipnctrieeonscdese  adsttshaa etthma fe im aefnilaedaln dgwa ewgomeormerkkneeetrnr p ttto ropooorckkoe  cttshheses,e s ass,og oalgelger gre errgseeapsgtpoainootsinnoibsoniilbf iotiryflei  rctoeyofc  rvoodifres dvwrsieie nrqweguq iainurnegidrd  eavmdne drmai fnvyaueinnarugilfa yddlia adnttaaga,t  edawn aehttnrialyte,r  wiyn ihtnoitleo   In the routine data management process, aggregation of records required manual data entry into  aisanp s rtpehiranIeIendn  atr s hdttohhhesu eeehrte  oreirtnouoe,euttua,i  tntndaiienndna eedtdda  ad d ditmagaait tigamataii n tzmmaiaaznagataainnetgoimaaeongmgneeoe enmomfntf toe e ponpnnrnerttoo e ppcrc reerereocsoscssocco  erettrdshshdssiis,t ,s   toaa roorgegoeskggkpsrrp 2oee2nogg mmsnaaistibtiniiiniobol.i.nni tlTTy  ioo thhwyffii   rssrawe esddcc aisoiogshgrr iaiddtstrihsisezz  dararaere ttbeqiqioeoduuntn wii brrpepeeerdedtronwo  mcm cefeeiaeasesnsnnlsd u uw wfawaiaellao s lddsd rraka retewtedaard  ou eeaurcnnnkcetdteerdr dyry  t  ioaitnnon 1ttdoo0     a spreadsheet, and digitization of one record took 2 min. This digitization process was reduced to 10  10Dasas   sMisnippn ODrrteethM.haa eIdedOn OssO .ht hDIeDenereKe mKttte ,,Sr s Saam cconnaasfddn  not   idfdwwm itigigooemiirr,tt kekiivz,zffl elavaoortetiwiwirfooiif..ncni acMM  oaotitffoooi  oorronneenn o ooeoevvf f r r eeppeerrcaac,,o poptthrerhedrdere   b  dtbtdaooaasaootsetakkeda d   v22rv e e rmemcreroicifiiirnfondci.r.cas  dTaT ttstihohoi ooitinsoskn  o dpd 2pkir0iggor  2soiic t0tcpeii zezsesassar ps ttrwi ieeowocranno asr  rpspegd crrg roofoerocrcaerdeeta tsls thyfssloey  waw r Dl attaaMlehsstree  Oerr redeDe, dd,dwMuua,h sccOaieelise,dnd   iw  ntttohoh t e i1h1l0e0e     OssiDOsn   iiKiD nnnth  K Sitttenhhch   SaOeteehcn   DOeaOOp nOKDDDr pDo KKKScrKc  oe SSaS scSccncsecaaa astpnnnhsnr     etwwopwhficrooeooeer rcrsflkkekdsiesf ffilsllwltood o itwtwow o wtr.oo..k   ooMkMMerk rk3 ooo3t errmo rmreeeo ooitoiknonvvv o teepephkrrreee,, ,r r t t t thsrhrhhoeeeeeecl ce  os odddorrrdaalaedet tstfaa  aporf   oreovvv srtneeehp rtsrrehoiiiif fbffeniiiiicc ceslfaaialiitdbtettyiii lioowodloinnnt ofwy   rppvp koorirererofoor kw.ccvc eTeeeiirsnehss.sswse gT    wwiwiahnnnaceaagrds ssi e  an agvggncsrerrerdeereedai aaavf ttctyslelloyyieyrnn d i asagfa uylcllttdtmioeeneanrrprgteeesat dduidd,o,,m,wan   aata pohasssf , it   iliiiwnenon n  ihtttn hhiholeeeef      thOtOOieinmDDDr oteKKKhtu iei  m tSnSS icnrc ectoaa aehinunnned   t  tppipavhnrtrereaeooor v ciccmdfeeeeirascsasistsafsnai   tcttat ihahhomgteeenieo a  m fffsnniiiete eeaselllntgddpdete   p wowwmp ofroo oefto rrhnrtckkhketeeee e spO rrrOs   rttDtDtoooohcKooKoiekskk  sSS   rsttccteh hhaatseneehnp    i sisosissooso n b llrlbseeeeeie c  sbrrracpeeieualssosiuspptpneysoo oseonnwin fbo ssstaifiihilb bsbitetihii syllaliheii pttt wyayyapr  p  aooewopsdfffo     vvsvwrbhikiieeeoefatwlwwrwrokewiiienfdnnl egogi gnnb w  avaaefinon ntiewlddndvl dv ie vvnveoeegwenlrr vri ioiidfifffrnyiyeyekngiiilnenntd iirgggfd i a w cedddnanaoaatditttrtoiaaakfDn,,i,e  c  wMwwra thhaOhioiniil.llndeee      in thaen dr oruecttiinfiec adtiaotna  omf tahnea mgeismreecnotg npirzoedce mssa cthhiinse r‐reesapdoanbsleib dilaittay  fiweladss  sbhya mreodv inbge tpwoeinetn‐b fyi‐epldoi nwt.o Trhkee r and  InaiDinntneM  dtrthhm Oreees.   crrIotonoifufu itttcteiiiamrnnmteeieo  s,ddn voa aeoftt arafti  i fitmmmhceeaaa ,ntnm ivaaoeiggnsreeriomemfifcceeopanngtatitnp o pipenzrrr eooodbccfa  eepmsssaessa pd cttehhhrrieii snsbc  eoarr‐eresrdseseppdsaoo dtrnonaeosscbiikolbberii2 dlldi0isttay ystt  oapwwo efakaires s2rl  des0shchs so aab rrprydeee ddrmf   orbboreeevctttohiwwnregdeeDe e pfnnMoo  rffi Oiniteeht,ll‐dedwb   yDwhw‐MipooleorrOkikinne,e trwr.t   haTahennhilddee     DMOO.D IKn  Stecarmn psr oofc etsims per,o vveidreifdic aa tsiiognni foicf apnat ptiemr eb raesdeudc trieocno frodr st hteo odkat a2 0tr sa npseferr r aencdo radg gfroerg tahtieo nD sMteOps,;  while  OODDDinMDMK thKOOSe c.S.  aO IcInnnaD n pttKe erprr oSmrmcocesascs  neooss ffpsi  t ttrpiitommorcooeeevks,,  isvvd3 ieeetmrr dtiifof iiainocc aksapitt gei3ioor nmnnirf  eioionccffoa   pppnredaatr ppt fireoemerrrc  ebbot haarredessede fifdduoe  crrlrd teetichcowooenrr o fddfiroesskr l etd ttoorh .owoeTkk od h 2r2aek00te i asnsr   .tcpp rTreaeehrnra  esrsr efeeienccdrooc crarredodnan  dffssoo eaurrdg m t tgchhproeeetn  igDDsoauMnMtmiooOOpnf,,t  t swiwiomtehnhpei iollseef;     in thheo OwDevKer S, ctaimn ep rroecqeusirse idt  tfooork t 3h em viner pifeicra rtieocno radn dfo rf otrhme  ffieillldin gw owrekreer .i Tnchree aisnecdr eaass ecdo mcopnasruedm ptot ion of  inhitinnitomh  twtehheceev eoiv  nerOOerr etriDDhfis, pKeKcto aiv  nSmtSeidccoreaiainn nfngir sc  peptpaeqrrtrpooiuoocccionereefess sssdsstteh   eisitetp f o ott Ofooor to foDh kktteKhh   33reeo S   mmOucvtaiDiiennnnrK e ippif s dieeScabrrcat  earatrciene omacc niouoas rsrn bddeaae ng offceodofamrr ut  htfetshohneereet a mo  (ffpTfii  epeatlfhbldidwlele l  wiw oa2npr,oog kFprr flikk gwwoeeurwreor..re r TTe ki6n hhf)vl.iee ono  iwiclnnrv ceciirnraneesvgaaeossdildeev ddeia nn cscgto oi fincnidocssuaemuntmmiptoipapnfrittcieiaooadnntn id o otoonff     time in the verification step of the ODK Scan is because of the app workflow involving identification  retcatciiontmmirdfiree ec  riasiennptci  ototthihnnfeeidc o vivafnetetgirrohi ifpnfeiir ccmooaafctti iisetoorhsnensec e  ssosmtt geeoipnpsf  ri toozehfefc e dott hhrgmoeen u OiaOztciDeDhndKieKn  m d eSS-acacrtaaceanhna m id iinssaae  bbbn‐lreeaeeccgaadaeduuamasstaebee  lnofioetffe   d (lttdThhasaetea b baa lfyppeie ppm2l  d,ww oFsvoo ibgirrnykukg frfmlleopo wo6wo)vi . ni iinnnt-vgvbo oypll-vvopiiinonngigtn‐  biitdd.yeeT‐npnhtoteiiiffnOiicctaDa. ttTKiioohnne    and rectification of the misrecognized machine‐readable data fields by moving point‐by‐point. The  ScaOaannnDddpK  rrr eoeScccctteaiisffnisicc ppaatrrtioiooovcnnei d sooseff d p Ttthrahaobeesvl  iemmgi d2ni.ie sisIfidrmreec pacacao onscggtti gnontinfiim zzOifeeeDiddcKra  em mnSdctaua atcnccihm htoiiionnen nv eera‐‐ferrrodieeorauautddsch tapaeibrobdolnlceaee  t fsdadoseaartsr tt taaaoh fn f efDsii eefdaelltaddart ssMaa  n bbatdnryyaa  agmnmgesmgoofervveerniig nnta.ag gnt  dippo oonaiignsngttter‐‐pbebgysy;a‐‐phptioooowiinnn etts.v.t  eTeTprhh,see;    ODK Scan process provided a significant time reduction for the data transfer and aggregation steps;  timOOhoDeDwKKree  qvSScuecarai,rnn e  tpdpimrrfoooeccr eesrstTessh q aeppburrlviooerev ve2ridi.idd fiI emecfddoap  traaai co  ststnh iioggefann  OniivffdDiieccKrRafaio onfnSiurtctcm t aatitninitmmie fioo Delennl  a i rrvnteeaaagd dnriuudowccu tetsfiir ooopernnrmo i fnfco oecrrfsr i setltehlhaisense oe  gddfd  aDawttaaaaste  attrcr reMaoa nnmaisnnTspffaiceemagrrreree  aeama Cndnseoeddnndt  otsaa .u ggamcggsoer rrdeerc ggeionasam pttiipooonannrd  sesittdnee gpptsso;;    prhhhcooooocrwwwerseeeessvvvepeseeorrroD,,n,   fadtttttiiaiihmmmn Megeeera    ponrrruareeegotqqqeicnmuueueisieirrrsndeeeetdda dsPt    roaoffffocomo etrrrhs  a sentett hhshar eeoeg   u eMvvvmtieaeenenrrRreiniaiffo fgitdiiucecc(amataaTittttnaeaiiinooeb omt nnlnD eP  a ra2anaoat,ncananeF dgddsise  g  mffufooorerrOrenmmmD6t   ) K(.fTff iiSilallclllbiiainnnnlegg gP   2 ro,www cFeeeeisrgrrseee u   riieinRnn 6ceccdr)rr.eeee Tasaaiisssgmeeendddeed    C aaPaossrsno   scccceuooosmmsmm epppdaaa irrrneee ddd   tttooo    cccooorrrrrreeeDssspppaootoannn MdddFiiiaonnnnrgggma   pgp pFerrirlmoooliccnceeeegnss st(sssp Peeeesrssro    fooocoffefr   msttthshh)e eees    rrr ooouuMutttiiainnnneeea   g1ddd emaaamtttiaanae    nm2mm7t  aaaPs nnnr oaaacgggeeeesmmms eeennnOttt   (((D2TTT Kmaaabbb iSnlllee ce5 a  7222n ,s,,     PFFFriiigoggcuuuerrrseees    666)))...   Redes igned Process  DDaFatatoa DrMA mCagtaoTa gFn lrrCliaaeelongglcTilsaetnlfatieTTmTTegiobcoraa tnaa len(ib(bbbeop cn ((llnollcep2eteee mo r.(e  P   2c2m22rfIpor. .om.. r ol  mpIIereIImcmpmmmtclpeeeoapl)pstpp)re ce sdaataat)eec)ccc o )stttt   f    oooo Offff    ODOOODKDDDRKKRKKoS uc o  1SSSS3a13t uccm––cc–in2anat24aa4 inmnoi  nne nddnd n   ioeaaoD oona2yy nvnnny D7ass a   s   vtvvvsar a  aa iaat ora rrriui iioooosuuuupssss r   pppporrcrrooooeOccscceesDees3essss–Kssss32ee41ee3 o  ms s0ssSm–df     o4oiscaooDni f ayfnffd     nasDD DDa 5t aaPya7aatts rttMasaaa o     cMMaMMensaaaaasnnnng aaaaeggggmeeeeTTmemmminimmeeeetnn.nneetttt ..C  ..C     oonnss uu mmeedd iinn  DatDaaMtaa TMnVraaaegnrneisfamfigecaeret nm(ioctoenPn m(rptop ePcrlere rosteescce)eo ssrsde) s   MManaanRRRgaooeog1uumue–tttm22ieiinn0 nned eesetna    DPDtyD rsPaaao rttt caoaae  c sesss  OODDKK3  mSS3cci naam nni nPP rroocceessss  RTRTTeiieimmdmdeeeeess    CiCCiggooonnnnneesssdd uuu PmmPmrreoeeoddcdce  e iiissnnnss     DDaaAttFagao  gMMrrmeaag nnFaaaitlgigloieennmmg ( eep(npnetetr   rPPr efrroocoroccmeredss)ss)  ee ss     MMMaaannnaaagg1gee em2mmm mieenennn i2nttt 7   PPP srrr ooo ccceeessssss    OODDKK2   SSmcc1aia0nnn  s 5  PP7rr oso cceessss   RRReeedddeeesssiiigggnnneeeddd     PPPrrroooccceeessssss    FoDrVFmFFaToeotoahrrrFri mmemiCfli lrc  o iFFeFanlidiltigllelillelociii(snnntnpiiggg goe(  n pnr(((ppep ef(oderece rrr orrd m  efmffaooco)troprrammmrl edm))t))  e  a) n agemen1t1 11mp3   mmrm–ion42iiicn 0nnd2e    s7s2a22 s7y77s   ss sssa     v   ed resources2222 3 b  mmmm–3y4 ii iminen nndl   ii55a55nm77y77   is sssns    ating tasks that dem      anded  DtaimtDDaDeaaCa atttoaaTan l  drCClCea oomconltllslilolefoeeenccncrtett iii(y(ooocc.nnon oI  mn m(((cc ctpopoohlmmlmeee ttprppeeoll)l)eeue tttteeei)n))   e data ma3n–3313a4––––g4424de    ddamddyaaaaeysyyynsssst      p   rocess, paper333‐3b––––3a4444 s  m dddedaadaianyyy yr sssse   cords were sent to th e    office  foTrhT Ardeagi nTrgTTgesirrrrtfdaaaieeznnnegrassssa(tfffitcieeegioororrnn m  n((( eccc vpo(doopilmmma eed trppplea ol)rllteeceeatattce eelom) )) r c dao)nu ra igere mseervnit1c ep–111.2––r –22Ho22d   m cddaodeyawaiasnysyyse sss v  s e  a rv iend t rhees roeudrecseig33s33 n 1  mmbmm0eyd iiiisnn nn  e p  lrimociensas,t imngan tuaaslk esn tthrya     to df etmhea nded  timeAr eagcngAoAAVdrrgeggd emggggrsarir rfoeaeteiiggtcngo aaaaenttthttyiiii(oeooo.p  nnnnIoen    r(f(((pfp pptrieheceeerercrre    o rr rrirreeseeodc ccccouo)ooorrrtrmdddidn))))p   e   l   edtaeltya  meliamni2an222mga  2 mmtem0iemn idiisnnn  e  a nntd p droatcae siss ,s penatp teo3r11 11‐ t0m0b00h   assesisn s   se edr vreerc oerledcstr woneircea lslye nt    th troou thghe  office  for diVniegteriVViVrtfinieeeczerrraaitiiftf fticiiiicoccooaananntttini(i ooopevnnneci  tr a(((ipppv releeieotcrrry oc  rrr.are eedTlccc )ohoocrerrodd dur)))e  r d  i eers igsnerevdi cdea.t2 a02H22 0m00so   sssaw   neavgeemr einn t tphreo creesds e333o3s  f mmmimfgeiiirninnnnee  d d f lpexroibcielistys,  tom saunpuearvl    iesonrtsr yin  of the  recorreTdvhsi eeaw rt ientdhgee ds ioiggfinftiaeclde d  diasta act aao nmmdp aenlneaatbgelleeymd e etlhnimetm pin rtoao tcpeerdso sva isndadev  fededea dtrabe asicosku  siremcnemts  etbdoyi a tehteleilmy s. ien ravteirn ge lteacstkros nthicaatl ldye tmharonudgehd   The redesigned data management process saved resources by eliminating tasks that demanded  itnimteer nTTaehnhtede c   rrmoeenddoneneesseciiyggti.nnv Ieenitdd yt  h.dd eTaa thrtaaoe  u mmrteiaandnneeaa sdggigaeetmnmae eemdnn atdt  npapatrragoo eccmmeessaessn n sastaag pvvereemoddce  rerneestsss op,o puuroarrccpceeeesssr s ‐bb boyyaf  sfeeeellirdimme rdiien ncfaaloettrixidnnibgsg i  wtltiaatessyrkk etss os  tte hhsnuaattpt  t dodere etvmhmiesaa oonnrfddsf ieeicdnde     time and money. In the routine data management process, paper‐based records were sent to the office  rttfieiommvr ieede  awaignniiddnti g mzm adootiinngoeeintyya .v.l  I Iidnnaa   ttlthhoaeec a  arrnolo duuc otteiiunnnreaei b eddlraea tdstaae   trmmhveaiacmnneaa.  tggoHee mpomwreeonnevvtti  dpeprerr o oifncceeee stsdhssb,,e  app rcaaekppd eieemrrs‐‐imbbgaanessdeeeddida   rtrpeeerlccoyooc.r redd ssss  ,ww meerraeen  ssueeannlt t e ttnoo t ttrhhyee   ooofff fftiihcceee    for digitization via local courier service. However in the redesigned process, manual entry of the  frfooercr  oddriidggsii ttiaizzta attthiiooenn o  vfvfiiiaac  ell ooicsca acll o ccmoouuprrleiieetrer  lsysee rervvliiimcceei..n  HaHtoeodww eeavvneedrr   diinna  tttahh eies   rrseeeddneetss itigogn nteehdde   ppserrorovcceeerss sse,,  lmemcaatrnnouunaailcl  aeelnnlyttr rytyh  orooff  uttghhhee    records at the office is completely eliminated and data is sent to the server electronically through  rirneectceoorrrnddesst   aactto  ntthhneee  cootffiffviicicteey  .ii ssT  chcooemm reppdlleeettseeigllyyn  eeedllii mmdaiinntaaa ttmeedda  naaanngdde  mddaaettnaat   iipss r ssoeecnnetts  stt ooo  fttfhheeer e ssdee rrfvvleeexrri  beeillleeictcyttrr otoonn iisccuaaplllleyyr  vtthhisrroooruusg gihhn    internet connectivity. The redesigned data management process offered flexibility to supervisors in  irinnettveeirrennweeittn  ccgoo ndnnnigeeiccttatiilvv diittayyt..a  TT ahhneed  rr eeenddaeebssiilggendne etddh  eddmaat ttaao  m mparaonnvaaiggdeeemm feeenendtt  bpparrcookcc eeismsss m ooffeffdeeriraeetdde  lffyllee. xx iibbiilliittyy  ttoo  ssuuppeerrvviissoorrss  iinn   reviewing digital data and enabled them to provide feedback immediately.   rreevviieewwiinngg  ddiiggiittaall  ddaattaa  aanndd  eennaabblleedd  tthheemm  ttoo  pprroovviiddee  ffeeeeddbbaacckk  iimmmmeeddiiaatteellyy..       FigurFeig6u.rRe e6d. Reseidgenseigdnedda tdaatma amnaangaegmemenenttp prroocceessss iinn OODDKK SSccaann ppapaepre‐tro-‐tdoi-gdiitgali tsaylssteymst.e m. Theredesigneddatamanagementprocesssavedresourcesbyeliminatingtasksthatdemanded timeandmoney. Intheroutinedatamanagementprocess,paper-basedrecordsweresenttotheoffice   for digitization via local courier service. However in the redesigned process, manual entry of the Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system.      Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system  .  Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system.  FFiigguurree  66..  RReeddeessiiggnneedd  ddaattaa  mmaannaaggeemmeenntt  pprroocceessss  iinn  OODDKK  SSccaann  ppaappeerr‐‐ttoo‐‐ddiiggiittaall  ssyysstteemm.. FutureInternet2016,8,51 10of17 records at the office is completely eliminated and data is sent to the server electronically through internetconnectivity. Theredesigneddatamanagementprocessofferedflexibilitytosupervisorsin reviewingdigitaldataandenabledthemtoprovidefeedbackimmediately. 3.3. FeedbackonAppUsabilityandWorkflowModifications Thefieldworkerslearnedquicklyhowtousethesmartphonesandswiftlybecamecomfortable withtheapplication’sworkflow: mostofthemwerealreadyusingsmartphonesintheirpersonallives. Nevertheless,theimplementationofODKScanwasguidedbytheprinciplesofminimaldisturbance incurrentworkflowandgeneratingvalueforuserstoimprovethechancesofadoption. Face-to-face interviewswereconductedindependentlywitheachofthefieldworkersattheendoftheintervention period. Theseinterviewsweresupportedbyasemi-structuredquestionnaire,whichincludedhintsto guidetheinterviewertoensurethecompletenessofthefeedback. Theinterviewswereconductedby aresearchteammemberinthelanguageoftheenrolledfieldworkers(Urdu). Allinterviewswere digitallyrecorded,transcribedinUrduandthentranslatedintoEnglish. Theinterviewrecordings andthecorrespondingtranslationswerethencheckedbyanotherteammemberforconsistencyand completeness. Data is segregated based on the various steps of the app workflow for the ease of analysisandpresentation. 3.3.1. FormFilling Inbothroutineandredesigneddatamanagementprocessesusersarerequiredtofillpaper-based form. Theformdesignsusedinroutineandredesignedprocessesarecompletelydifferent,butdata fieldsarecommonbetweenthetwodesigns. Furthermore,theScan-compatibleTB03formallowed morestructuredandfewerhandwrittentextfields,whereasthestandardTB03formincludedhand written data fields only. The ODK Scan process begins with filling in the ODK Scan-compatible form. This includes marking the correct bubble for the fill-in bubble fields, linking guide dots for thestructurednumberfieldsandwritinginthetextfields. Fieldtestresultsshowedthatthisstep took more time as compared to the corresponding step of the routine data management process. Moreover, in terms of learning to fill in the new form, all of the field workers adapted to the formwellandfounditeasytolocatethedesiredfield. Nonetheless,almostallfieldworkerswere unhappy about the time-consuming aspect of the form filling for the ODK Scan process (Table 3), while at the same time they acknowledged the benefit of using number boxes and fill-in bubbles. OneDFSsaid,“Thisexperiencewasverygood. Thisisexcellent. Especiallydotsandbubbleshave eliminatedanychanceofthedatamanagementofficercontactingusforclarificationortoaddressany readabilityissue.” Interestingly,oneDFShighlightedadifferentaspectofthis,saying “[I do not like] filling the dots on the form as they are time consuming. Also, filling the dots embarrasses me whenever I do that in the presence of others. It seems like I am a kid and practicinghandwriting.” Table3.Self-administeredquestionnaireitems. LearningtoFill Locating Easeof Easeof FieldWorkers(FW) NewForm DesiredField WritingNumbers FillingBubbles 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 FW1 FW2 FW3 FW4 1correspondstoveryeasyand5correspondstodifficult.

Description:
mHealth Coordinator, Mercy Corps, Pak Palace, Murree Road, Rawal Computer Science and Engineering, University of Washington, Seattle, WA 98101, USA; programs mostly maintain and manage paper-based patient recording and reporting systems [6]. resource for front line workers [10].
ebook img

ODK Scan PDF

17 Pages·2016·6.25 MB·English
by  
Save to my drive
Quick download
Download
Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview ODK Scan

future internet Article ODK Scan: Digitizing Data Collection and Impacting Data Management Processes in Pakistan’s Tuberculosis Control Program SyedMustafaAli1,*,RachelPowers2,JeffreyBeorse3,*,ArifNoor1,FarahNaureen1,*, NaveedAnjum1,MuhammadIshaq1,JavariyaAamir1andRichardAnderson3 1 mHealthCoordinator,MercyCorps,PakPalace,MurreeRoad,RawalChowk,Islamabad44000,Pakistan; [email protected](A.N.);[email protected](N.A.);[email protected](M.I.); [email protected](J.A.) 2 InformationSystems,VillageReach,Seattle,WC98102,USA;[email protected] 3 ComputerScienceandEngineering,UniversityofWashington,Seattle,WA98101,USA; [email protected] * Correspondence:[email protected][email protected](S.M.A.); [email protected](J.B.);[email protected](F.N.);Tel.:+92-30-1855-0454(S.M.A.); +1-206-221-7701(J.B.);+92-30-2850-6178(F.N.) AcademicEditor:DinoGiuli Received:30August2016;Accepted:12October2016;Published:24October2016 Abstract: Thepresentgrievoustuberculosissituationcanbeimprovedbyefficientcasemanagement and timely follow-up evaluations. With the advent of digital technology, this can be achieved throughquicksummarizationofthepatient-centricdata. Theaimofourstudywastoassessthe effectiveness of the ODK Scan paper-to-digital system during a testing period of three months. Asequential,explanatorymixed-methodresearchapproachwasemployedtoelucidatetechnology use. Training,smartphones,theapplicationand3G-enabledSIMswereprovidedtothefourfield workers. Atthebeginning,baselinemeasuresofthedatamanagementaspectswererecordedand comparedwithendlinemeasurestodeterminetheimpactofODKScan. Additionally,attheendof thestudy,users’feedbackwascollectedregardingappusability,userinterfacedesignandworkflow changes. Atotalof122patients’recordswereretrievedfromtheserverandanalysedintermsof quality. ItwasfoundthatODKScanrecognized99.2%ofmultiplechoicefill-inbubbleresponsesand 79.4%ofnumericaldigitresponsescorrectly. However,theoverallqualityofthedigitaldatawas decreasedincomparisontomanuallyentereddata. UsingODKScan,asignificanttimereduction isobservedindataaggregationanddatatransferactivities,butdataverificationandform-filling activities took more time. Interviews revealed that field workers saw value in using ODK Scan, buttheyweremoreconcernedaboutthetime-consumingaspectsoftheuseofODKScan. Therefore, itisconcludedthatminimaldisturbanceintheexistingworkflow,continuousfeedbackandvalue additionsaretheimportantconsiderationsfortheimplementingorganizationtoensuretechnology adoptionandworkflowimprovements. Keywords: mHealth; ODK scan; mobile health application; digitizing data collection; data management processes; paper-to-digital system; technology-assisted data management; treatmentadherence 1. Introduction Tuberculosis(TB)continuestobeaglobalandgravepublichealthconcern. Itisahealthissue formillionsandthecauseofnumerousdeathseachyearworldwide[1]. Despitetheavailabilityof effectivetreatmentsandstrategiestoimproveaccesstotreatmentregimens,multidrug-resistantand FutureInternet2016,8,51;doi:10.3390/fi8040051 www.mdpi.com/journal/futureinternet FutureInternet2016,8,51 2of17 extremelydrug-resistantTBcasesareemergingduetothelackofadherencetotreatmentprotocols. Thisnon-adherenceismainlybecauseofthelongertreatmentcourse(6to8months)andunpleasant sideeffectsofthedrugtreatment[2]. PakistanisaTBhigh-burdencountry,experiencingtheemergenceofalargenumberofnewTB cases(approximately420,000everyyear)andprevalenceofmultidrug-resistantTB.Pakistanranks fifthfordrug-susceptibleTBandfourthformultidrug-resistantTB[3]. Theburdenofdiseasecaused byTBandtheeffectivenessofdifferentprogrammaticinterventionsarebetterunderstoodthrough an interpretation of the available epidemiological data [4]. The program management function, Monitoring and Evaluation (M&E), makes the best use of the available resources (including data) to guide and support programmatic decision-making. The M&E system plans and conducts data collectionandanalysistoinformprogrammanagersandkeepperformancemeasuresontrack[5]. Unfortunately,thisimmenseresourceiseitherunderutilizedornotusedatall[4]. Health information systems used in disease-specific health programs are designed to collect patient-centricdatatogenerateactionableinformationtoguidedecision-making. Likewise,TBcontrol programsmostlymaintainandmanagepaper-basedpatientrecordingandreportingsystems[6]. Fidelity to treatment protocols is ensured through case management (DOTS strategy) and follow-upevaluations(sputumspecimenformicroscopicexaminationorcultureatspecificpointsin thetreatmentperiod)[7]. Generally,intheTBcontrolprograms,paper-basedpatientrecordsareused toprepareperiodicreportsonthepatientenrolment,retentionandtreatmentprogress. Thesedata summariesarealsosenttodifferentmanagementlevelsforaggregation,analysisandreporting[6]. However, paper-based recording and reporting systems suffer from some inherent challenges, whichinclude:checkingdataquality,dataentryandaggregation,allottingtreatmentoutcomecorrectly, fillinggapsindata,analysingdataandgeneratingactionableinformation. Thesedatamanagement activities are inefficient, work-intensive and time consuming [8]. However, the advent of digital technologyhaspositivelyimpactedtheorganizationalworkflow[9]. ElectronicHealthInformationSystems(eHIS)havegeneratedbetterresultsonthethreefrontsof TBcontrolprograms: patientcare,resourcemanagementanddiseasesurveillance[8]. Increasingly, mobilehealth(mHealth)interventionsaregrabbingtheattentionofthepublichealthprofessionals becauseoftheirabilitytoimprovethepublichealthsituation[10]. mHealthreferstotheapplication of portable hand-held devices and communication capabilities in the health care programs [11]. Duringthepastdecade,mHealthhasbeencontinuouslyevolvingwithintheeHealthdomain,whereit hasshowngreatpotentialtoovercomemanytraditionalbarriersindeliveringcareandreducingthe associateddistance,timeandcost[12]. Fortunately,mostdevelopingcountrieshavealreadylaunched mHealthinitiativestounleashthepowerandreachofsmartphonesandconsiderthemanimportant resourceforfrontlineworkers[10]. Early diagnosis and vigilant management are important considerations to successfully control TB. Therefore, in TB control programs, case management involves a multidisciplinary approach for comprehensive and coordinated follow up of the presumptive and confirmed TB case[13]. Thisapproachincludesimprovedcommunicationbetweenpatientandcaregivingteams, patienteducationanddeliveryofsupportduringtreatment[14]. Increasingly,mHealthapplications arebeingusedtosupporthealthcareteamsinearlydiagnosisandmonitoringpatientprogress[15]. ManyoftheexistingmHealthsolutionsforTBcasemanagementarecustomsoftwaredevelopedfor individual organizations and use cases, they are not freely available for other organizations to try. TherearealsogeneralpurposemHealthsolutionsavailable,suchasMedicMobileorODKCollect anditscousins. Thesearedirect-to-digitalsystemsthatdonotinvolveanypaperrecordkeepingor paperarchives. Among the various reported uses of mHealth tools the most reported are: data collection, electronicdecisionsupport,provider’seducationandtraining,andcommunication[10]. However, theparticularuseofmHealthfordatacollectionhasagreaterlikelihoodofimprovingpublichealth in low- and middle-income countries [16]. Noteworthy are those unpredictable implementation FutureInternet2016,8,51 3of17 challengesandcomplicatedsituationalfactorsthataffectanintroductionofmHealthapplicationsin primaryhealthcaresettingsofthedevelopingworld[17,18]. Sincethereareanumberoffreely-availableopensourcemHealthapplicationsaimedatimproving datacollection,onlyonetoolleveragedthestrengthsoftheexistingpapersystemwiththeflexibility and efficiency of the digital records. We chose the ODK Scan paper-to-digital application to test improved data collection in the public–private mix model of the TB control program of Pakistan. TheultimategoalofthisinterventionwastoreducethetimeandresourcesspentonTBdatacollection byimprovingthedatacollectionanddigitizationprocessandreducingrelianceonmanualdataentry. Additionally,ODKScanwaschosenbecauseitallowssupervisorsandmanagerstoassessthedata qualitybyreviewingtheoriginalimagesnippets. Thespecificobjectivesofthispilotstudywereto: (1)assessdataqualityaspectsatboththeappanduserlevels,(2)assessthequantitativeimpactof ODKScanonthedatamanagementprocesses,and(3)collectfeedbackonapplicationuserexperiences toascertainappusability,userinterfacedesignandworkflowchanges. 2. MaterialandMethods 2.1. DescriptionofTechnology OpenDataKit(ODK)ScanisanAndroid-basedmobileapplicationandapartoftheODK2.0 suiteoftools[9,19]. Itusesthedevice’sbuilt-incameratocaptureanimageofascan-compatiblepaper form, which it feeds into a series of image processing algorithms (SURF, fast approximate nearest neighbor,andRANSAC)toestablishapoint-by-pointmappingbetweenthetemplateandthecaptured image [20–23]. Using this mapping, Scan segments the form into small image snippets that each containsasingledatafieldfromtheoriginalform[24]. Theimagesnippetsfallintothreepossible prompttypes: hand-filledbubbles,structurednumberboxes,andQRcodes. Hand-filledbubblesare multiple-choicequestionsthatarecompletedbyfillinginthebubblethatcorrespondswiththedesired response. Scanclassifieshand-filledbubblesusingaSupportVectorMachine(SVM)andprincipal componentanalysis(PCA)[20,25,26]. Structurednumberboxesarea2×3gridofdotsthatcanbe connectedbystraightlinestodrawnumerals. ScanclassifiesnumbersusingaMulti-layerPerceptron (MLP)withbackpropagationoferror[27]. Finally,QRcodesaretwo-dimensionalbarcodesthatcan beprocessedasalphanumericstrings. ScanusestheZebraCrossing(Zxing)librarytoclassifyQR Codes[28]. AfterScanhasclassifiedeachprompt,itinsertsthedataintoaneditablerecordintheODK 2.0database.Eachdigitizedfieldincludesacopyofitsassociatedimagesnippetfromtheoriginalform, whichcanbeusedtoverifydigitizedfields. OncetherecordisintheODK2.0database,userscanuse theothertools,suchasODKSurveyandODKTablestoview,edit,verify,aggregate,andsyncthedata orcreatecustomizedservices,interactionsandreportswiththedata[19]. ODKScandoesnotprocessfreehandwrittentext. Promptsthatrequirewrittentextwillinclude boxes on the paper forms that users can fill with their hand written text, but Scan will leave the correspondingfieldinthedatabaserecordblank. Handwrittentextfieldscanbemanuallytranscribed onthedevicebyviewingthepromptinODKSurvey,whichwilldisplaythecorrespondingimage snippet, and typing in the text via the phone’s keyboard. Alternatively, they can be manually transcribedonaPCaftertherecordhasbeensynced,againbyviewingtheimagesnippet. Scan and the rest of the ODK 2.0 tools do not require a consistent Internet connection to function[19]. Alloftheimageprocessingisperformedbythedeviceitself,andthedataisstoredina localdatabase. AnInternetconnectionisonlyrequiredforsyncingrecordswiththecloud,andthis canbedoneattheleisureoftheuser. 2.2. ODKScanWorkflow BeforeODKScanistobeused,ascan-compatibleformmustbedesignedusingtheformdesigner. Paper copies of the form image are printed out and the form’s definition files are copied to the Android-baseddevice. Aftertheformisfilled,theusertakesapictureusingthebuilt-incameraof FutureInternet2016,8,51 4of17 Future Internet 2016, 8, 51   4 of 16  of the smartphone. ODK Scan detects the data fields and auto‐digitizes them. The user can then  thesmartphone. ODKScandetectsthedatafieldsandauto-digitizesthem. Theusercanthenlaunch launch ODK Survey to view and verify data fields, and editing if necessary. After a user is sure about  ODKSurveytoviewandverifydatafields,andeditingifnecessary. Afterauserissureaboutthedata the data quality, the patient record is synced with an aggregate server. If no internet connection is  quality,thepatientrecordissyncedwithanaggregateserver. Ifnointernetconnectionisavailable,the available, the user can delay synchronization and continue collecting data until connectivity is  usercandelaysynchronizationandcontinuecollectingdatauntilconnectivityisrestored. Atanytime, restored. At any time, the user can view all synced records on the smartphone and run customized  theusercanviewallsyncedrecordsonthesmartphoneandruncustomizedreportstoviewactionable reports to view actionable information. For this trial we created a report that generates information  information. Forthistrialwecreatedareportthatgeneratesinformationaboutpatientswhosefollow about patients whose follow up appointments are either overdue or due in the coming days. Aside  upappointmentsareeitheroverdueordueinthecomingdays. Asidefromthesyncingstepofthe from the syncing step of the ODK Scan workflow, all steps can be undertaken without an internet  ODKScanworkflow,allstepscanbeundertakenwithoutaninternetconnection. connection.   2.3. Intervention 4. Intervention   AmongninerecordingandreportingformatsusedintheTBcontrolprogramofPakistan,one Among nine recording and reporting formats used in the TB control program of Pakistan, one  recordingformat(TB03)wasselected(Figure1). Itsscan-compatibleformwasdesignedusingthe recording format (TB03) was selected (Figure 1). Its scan‐compatible form was designed using the  ODK Scan Form Designer to create machine-readable fields for the app to recognize (Figure 2). ODK Scan Form Designer to create machine‐readable fields for the app to recognize (Figure 2). The  The scan-compatible form consists of three types of machine-readable data fields, i.e., structured scan‐compatible form consists of three types of machine‐readable data fields, i.e. structured number  numberboxes,fill-inbubblesandQRcodes,andonenon-machinereadabledatafield,i.e.,handwritten boxes, fill‐in bubbles and QR codes, and one non‐machine readable data field, i.e. hand written text  textfields(Figure3). Aftertheformdesignwasprepared,itwaspre-tested,necessarymodificationsin fields (Figure 3). After the form design was prepared, it was pre‐tested, necessary modifications in  formdesignweremade,andtheformswereprintedandboundintoaregisterbook. Theintervention form design were made, and the forms were printed and bound into a register book. The intervention  started at the beginning of May 2016 and lasted for three months in district-level field settings. started at the beginning of May 2016 and lasted for three months in district‐level field settings.  Informed consent was sought from four of the field workers, belonging to two enrolled districts Informed consent was sought from four of the field workers, belonging to two enrolled districts  (ChakwalandHafizabad)oftheTBcontrolprogramofPakistan. Thefieldworkers(DistrictField (Chakwal and Hafizabad) of the TB control program of Pakistan. The field workers (District Field  SuSupperevrvisisoorrss)) wweerree ttrraaiinneedd iinn uusisningg thteh eScSacna‐nco-cmopmaptiabtlieb TleB T0B3 f0o3rmfo arnmd athned atphpe waporpkfwloowrk aflnodw waenred  wgeirveeng ivsemnasrmtpahrotnpehso n(Seasm(Ssaumngs uGnaglaGxayl aJx7y aJ7nda nHduHauwaewi eYi6Y) 6a)nadn dscsacna‐nc-ocmompaptaibtilbe leTTBB030 3rreeggisitsetersr.s . AAdddidtiiotinoanlalyll,yw, wepe rporvoivdieddedth tehmem3G 3-Ge‐neanbalbedledm moboibleilSeI SMIMsfso froirn itnetrenrentecto cnonnenceticvtiivtyitayn adndm monotnhtlhyltyo tpo-pu‐p fourps hfoarr isnhgadriantga vdiaatain vteiar ninett.erWneeti.m Wpeo simedpnooserde sntroi crteiostnriocntioancc oenss aincgceosrsidnogw onr ldoaodwinnlgoaandyinogt haenrym oothbeilre  apmpolbicialeti oanpsp.liFciaetlidonwso. rFkieelrds wweorrekaerllso wweedret oalulosweesdm atort puhseo nsmesa,rintpchluodniensg, imncalkuidnigngca mllsa,kteinxgt mcaelslss,a gteixntg  anmdesinsategrinnegt abnrdo winstienrgn.et browsing.     FFigiguurree1 1..O Orriiggiinnaall TTBB0033 ffoorrmm uusseedd iinn rroouuttiinnee ddaattaa mmaannaaggeemmeenntt pprroocceessss.. FutureInternet2016,8,51 5of17 Future Internet 2016, 8, 51   5 of 16  Future Internet 2016, 8, 51   5 of 16      Figure 2. Scan‐compatible TB03 form used in ODK Scan paper‐to‐digital system displaying fill‐in  Figure2. Scan-compatibleTB03formusedinODKScanpaper-to-digitalsystemdisplayingfill-in Fbiugbubrlee s2, .n Sucmanb‐ecro bmopxaetsi balned T tBex0t3 f ifeolrdms.  used in ODK Scan paper‐to‐digital system displaying fill‐in  bubbles,numberboxesandtextfields. bubbles, number boxes and text fields.  (a)  (b) (a)  (b)   (c)  (d)   (c)  (d) Figure 3. Formats of data fields on ODK Scan‐compatible TB03 form. (a) Machine‐readable structured  Fniugmurbee r3 . bFooxrems;a t(sb o) f mdaatcah fiineeld‐rse oand aObDleK f iSlcl‐ainn ‐cboumbbplaetsi;b l(ec )T Bu0s3e rf otrrman. s(car)i Mbeadc hteinxet ‐rfeiealdda; b(lde) s tmruaccthuirneed‐  Figure3.FormatsofdatafieldsonODKScan-compatibleTB03form.(a)Machine-readablestructured nreuamdabbelre  bQoRx ecso; d(eb. ) machine‐readable fill‐in bubbles; (c) user transcribed text field; (d) machine‐ numberboxes;(b)machine-readablefill-inbubbles;(c)usertranscribedtextfield;(d)machine-readable readable QR code.  QRcode. FutureInternet2016,8,51 6of17 2.4. Design A sequential, explanatory mixed-method research approach was employed to elucidate the technologyuseforaneffectiveimplementationoftheODKScanpaper-to-digitalsystem. However, themixedmethodresearchapproachinvolvescombiningthequantitativeanalysiswithqualitative reasoning. Thisresearchapproachincludesapplicationofthreemethods: (1)quantitativeassessment ofthedataqualityatbothappanduserlevels;(2)quantitativemeasurementofthetimespentinfilling forms,datatransfer,aggregationandverification;and(3)qualitativefeedbackfromthefieldworkers (users)tocapturetheircompleteexperience. However,eachofthesemethodsisexploredinmore detailbelow. (1) Quantitative assessment of the data quality was done at both the app and user levels. Attheapplication level, accuracy of ODK Scan’s ability to recognize fill-in bubbles and number boxeswasmeasuredbycomparingtherawScanvalues(thedigitizedvalueswhichScan’salgorithm producedbeforehumanverification)withthedataappearinginimagesnippetsofthecorresponding data fields. To calculate this, the team hired a Seattle USA-based data analyst to examine each individual number and fill-in bubble image snippet, to record the observed “ground truth” value as observed from the image, and to compare it with the corresponding raw value that ODK Scan processing recorded. Comparison of these field value pairs allowed us to calculate ODK Scan’s accuracyratesforallthenumberandfill-inbubblefields. At the user-level, the workflow allowed users to improve the quality of the data through verificationandeditingdatafieldsmanually. User-leveldataqualitywasassessedbycomparingthe same“groundtruth”valuesforeachfieldtothefinalvaluesthatthefieldworkersyncedwiththe server. Theseaccuracyrateswerepredictedtobehigherthantheapp-levelaccuracyratebecausethe fieldworkershadtheopportunitytocorrecterrorsfromthe“raw”Scanprocessing. Furthermore, we calculated the accuracy rates of the existing (non-ODK Scan) manual entry processesbycomparingthefieldvaluesobservedfromthehardcopyTB03registerpagestothevalues manuallyenteredbythedataentryworkerintheMicrosoftExceldatasheets. Thiswouldprovide us with a baseline for manual entry accuracy rate. Accuracy rates for both user-level digitization processes were calculated based on the full field value, rather than by individual digit or bubble (binaryassessmentofcorrectorincorrectforeachfield). (2)Quantitativemeasurementsofthetimespentinfillingforms,datatransfer,aggregationand verificationwererecordedintheroutinedatamanagementprocessusingdirectobservation.Theywere thencomparedwiththeendlinemeasuresforthesamedatamanagementprocessesusingtheODK Scansystem. (3) Gathering of qualitative feedback on app use experience was done through surveys and interviews. The survey captured Likert-scale assessment, while face-to-face interviews using a semi-structuredquestionnairewerecarriedoutwitheachofthefieldworkers. Theirexperienceswith appusabilityandworkflowmodificationwereexploredtoassesstheeffectivenessoftheODKScan paper-to-digitalsystemfromausabilityperspective. 3. Findings 3.1. DataQuality App-level quality measure: During an intervention period of three months, 122 patients’ paper-basedrecordswerescanned,digitizedandcheckedforquality.Allofthesedataweresyncedand availableonthecloud-basedODKaggregateserverforaccessandremoteanalysis.Thefirsttaskwasto assessODKScan’sabilitytocorrectlyparsetheuserfilledmachine-readabledatafields,thestructured numberboxesandthefill-inbubbles. Accuracyratesforthesefieldswerecalculatedbymanually reviewingthe122records,consistingof10,600digitsand8000fill-inbubbles,andcomparingScan’s rawpredictedvalueagainstthegroundtruthvalueoftheimagesnippet. Therecognitionaccuracyfor thefill-inbubbleswasfoundtobemuchhigherthanthatofthenumberboxes. Includingblankfields FutureInternet2016,8,51 7of17 (fieldsthatusersdidnotfillin),theScanappclassified99.2%fill-inbubblesand79.4%digitscorrectly (Table1). Table1.Measurementofaccuracyratesforrecognitionoffill-inbubblesandnumberboxes. Future Internet 2016, 8, 51   7 of 16  Table 1. MeasTuyrpemeoenftF oife aldccuracy rAatlelsF fioerl drescognitioBnl aonf kfillF‐iine lbdusbEblxecsl uandde dnumber boxes.  Number 79.4% 77.6% Fill-TinypBeu obfb lFeiseld  9A9l.2l %Fields Blank Fie9l7d.2s% Excluded  Number  79.4%  77.6%  Fill‐in Bubbles  99.2%  97.2%  The misclassified number fields were further investigated by mapping accuracy measures of thenumberboxesontheTBformtospecificfieldsonthephysicalpaper. Itwasfoundthatfields The misclassified number fields were further investigated by mapping accuracy measures of the  locatendumobnerth beoxreisg hont tshide eTBo ffotrhme top aspgeec,iftihc efiesliddse ofna trhteh epshtyfsricoaml pathpeerr. eItg wisatse rfobuonodk th’satb fiinelddisn lgo,castuedff ered fromolno wtheer riagchctu sriadcey ofm theea spuagrees, tthhea snidteh fearnthuemst bfreormb othxee srewgishtiecrh bowoekr’se blioncdaitnegd, sculfofesreerdt ofrotmhe locwenerte r or left-haacncdursaicdye m(eFaisguurrees t4h)a.nI nthsep neuctmiobnero bfotxhees wcahpicthu rwederei mloacagteesd schlooswere tdo tthhea tcetnhteerp oarg leesft‐whaenrde sbiedne ding away(Ffrigoumret h4)e. Ibnisnpdeicntigo.nT ohf itshbe ecnadptiunrgedw iamrpageeds tshheowpeodin tth-atot -tphoe ipnatgmesa wpperien gbeonfdtihnge caawpatyu frreodmi mthaeg eto binding. This bending warped the point‐to‐point mapping of the captured image to the template file,  thetemplatefile,whichcausedtheexpectedimagesnippetcoordinatestomismatch. Theresulting which caused the expected image snippet coordinates to mismatch. The resulting image snippets  imagesnippetsweremisaligned,meaningthattherewouldbemissingedgesandotherimportant were misaligned, meaning that there would be missing edges and other important information. The  information. The structured number classifier’s accuracy suffered when attempting to classify structured number classifier’s accuracy suffered when attempting to classify misaligned image  misalignedimagesnippets. snippets.    Figure 4. Mapping of the accuracy measures of the number boxes.  Figure4.Mappingoftheaccuracymeasuresofthenumberboxes. User‐level quality measure: The ODK Scan workflow includes viewing, editing and verifying  User-levelqualitymeasure: TheODKScanworkflowincludesviewing,editingandverifying data before it is synced with the server. Field workers were therefore given complete responsibility  databfoerf othree iqtuisalsityyn acsesdeswsmitehntth aensde rimveprr.oFvieemldenwt.o Hrkoewrsevwere,r eint htheer eefxoirsetinggiv menancuoaml dpaletate ernetsryp opnrosicbeislsi,t yfor thequthailsi rtyesapsosnessibsimliteyn its ashnadreidm bpertwoveeenm ae fnietl.dH woowrkeevr earn,din a tdhaetae mxiasntianggemmeannt oufafilcdera (tDaMenOt)r.y Thper oDcMesOs, this respoins sriebsipliotynsiisblseh aforre ddabteat weneteryn aanfide lqduwaliotyrk aetr tahned naexdta lteavmel aonfa mgeamnaegnetmoefnfitc. eWr(hDenM dOa)t.aT qhueaDlitMy Ois respo(nascicbulreacfyo)r odfa tsaynecnetdr yreacnodrdqsu waleirtey cahtetchkeedn eaxntdl ecvoemlopfarmeda nwaigthe mtheen tq.uWalihtye nofd amtaanquuaalllyit yen(taecrceud racy) records, it was found that the existing manual data entry process resulted in a data accuracy rate of  ofsyncedrecordswerecheckedandcomparedwiththequalityofmanuallyenteredrecords,itwas 94%, while the ODK Scan process only reached 86%. As described in the methodology section, both  foundthattheexistingmanualdataentryprocessresultedinadataaccuracyrateof94%,whilethe of these accuracy rates were calculated by comparing values observed from the paper records with  ODKScanprocessonlyreached86%. Asdescribedinthemethodologysection,bothoftheseaccuracy the values that were saved to the digital record in each process. While it is understandable that human  accuracy for data entry did not reach a perfect 100% for either processes, the discrepancy between FutureInternet2016,8,51 8of17 rateswerecalculatedbycomparingvaluesobservedfromthepaperrecordswiththevaluesthatwere savedtothedigitalrecordineachprocess. Whileitisunderstandablethathumanaccuracyfordata entrydidnotreachaperfect100%foreitherprocesses,thediscrepancybetweenthetworatesshows thattFhuetutrae sInkte-rdneetp 20e1n6d, 8e, n51c e  offield-workerfordataverificationwithODKScandidnotmeetth8 eof q16u ality oftheexistingmanualdataentry. ThisdiscrepancybetweenthemanualdataentryandverifiedODK the two rates shows that the task‐dependence of field‐worker for data verification with ODK Scan  Scandataaccuracyratesisfurtherexploredinthediscussionsection. did not meet the quality of the existing manual data entry. This discrepancy between the manual data  entry and verified ODK Scan data accuracy rates is further explored in the discussion section.   3.2. ImpactingDataManagementProcesses 7. Impacting Data Management Processes   In the TB control program, the routine data management process begins with the collection of data frIonm thael lTBen croonltlreodl pgreongerraaml, pthrae crtoituitoinnee rdsat(aG mPsa)n.agOenmceentt hperoccoensss obleigdiantse wditpha tpheer c-bolalescetdiorne opfo rt is prepadraetda iftroismt raalnl sefnerrorleledd togetnheeranl epxrtalcetivtieolnoefrsm (aGnPasg). eOmnecne tt(hseu bco-rnescoilpidieatnetdo pffiapcee)r,‐bwasheidc hretphoernt pisa sses throupgrhepmaruedlt iipt ilse tdraensskferrerevdie two sthaen ndexits lefivneal lolyf meannteargeedmeinntt o(suanb‐rEexcicpeilenstp orfefaicdes),h weehti.chT thheend pigasitsiezsa tion through multiple desk reviews and is finally entered into an Excel spreadsheet. The digitization of  ofpaper-basedrecordstakesasignificantamountoftimebeforeitissharedwiththenextlevelof paper‐based records takes a significant amount of time before it is shared with the next level of  management (principle recipient office) for finalization and progress review meetings (Figure 5). management (principle recipient office) for finalization and progress review meetings (Figure 5).  Timespentinform-filling, datacollection, transfer, aggregationandverificationwasmeasuredby Time spent in form‐filling, data collection, transfer, aggregation and verification was measured by  the direct observation. At baseline, these measures were recorded by observing the routine data the direct observation. At baseline, these measures were recorded by observing the routine data  managementprocess. However,attheendline,thesamemeasureswererecordedbyobservingthe management process. However, at the endline, the same measures were recorded by observing the  ODKOSDcaKn Swcaonr kwfloorkwfl.ow.     Figure 5. Routine data management process in the TB (Tuberculosis) control program of Pakistan.  Figure5.RoutinedatamanagementprocessintheTB(Tuberculosis)controlprogramofPakistan. Filling in the ODK scan‐compatible TB form took longer than the old form. Extra attention was  FillingintheODKscan-compatibleTBformtooklongerthantheoldform. Extraattentionwas required for filling in bubbles and writing structured numbers on the guiding dots correctly. Each  requifrieedldf owrofirklleinr g(ni n= b4u) bwbalse soabnsedrvwedri twinhgilest rfiullcintugr eind tnhureme bSecrasno‐cnomthpeagtiublied ifnogrmdso. tTshceo rarveecrtalyg.e Etaimche field worktearke(nn =to 4f)ilwl 1a2s Socbanse‐crovmedpawtibhliel efofirlmlisn gwains 0th2 rmeeinS 5c7a ns- c(momaxp, a0t3i:b04le sf; omrmins, .01T:h48e sa)v. eHroawgeevteimr, eont aken tofilla1v2erSacgaen it- ctooomkp 1a mtibinle anfodr m27s s wtoa fsil0l 2inm onine 5p7atsie(nmt raexc,o0rd3 :u04sinsg; mthien e,x0i1st:4in8gs T).BH03o rwegeivsteerr,. oTnhea nveexrta geit tooka1smpeicnt oafn ddat2a7 msatnoagfielmliennto, ndeatpa acotilelencttiroenc, owrdas unsoitn agffethcteede xbiys ttihneg OTDBK0 S3craeng wisotrekr.floTwh ebenceaxutsea sthpee ctof process remained the same: visit all of the enrolled GPs of a district. Finally, with the routine data  datamanagement,datacollection,wasnotaffectedbytheODKScanworkflowbecausetheprocess management process, once data collection was completed, it took 1–2 days to transfer data via local  remainedthesame: visitalloftheenrolledGPsofadistrict. Finally,withtheroutinedatamanagement courier service, in the form of hard‐copies. However, with the ODK Scan process the same transfer  process,oncedatacollectionwascompleted,ittook1–2daystotransferdatavialocalcourierservice, of data only lasted a few min. It only required the user to get to an area of internet connectivity and  intheformofhard-copies. However,withtheODKScanprocessthesametransferofdataonlylasted send the data to the ODK Aggregate Server. Future Internet 2016, 8, 51   9 of 16  FutureIFnutteurrnee Itn2te0r1n6e,t 820,1561, 8, 51   9 of 196 of17 In the routine data management process, aggregation of records required manual data entry into  In the routine data management process, aggregation of records required manual data entry into  aafFe uswtpurarmee  sIanipndtre.esrnahIetdet ose2hn0t,e1l e6ayt,n ,8r da,e n5q d1du i  igdriiegtdiiztiatzhtaietoiounn so eofrf o toonneeg  erreetcctooorrdadn  ttooaoorkek a2 2 mo mfiniin.n Tt.e hTrinsh edisti gcdioitginzinatteiizcoatnitv ipoirtnoy cpaersnosd cwesasessn  rwdedathus cereeddda tutoac 1et0do  9tt oho ef1 106   OFFsDFu uuiKtttnuuurr rsAeete   h IIiIgnnnnetgtt  eeetOrrrrhnneneDeeget tt O a K2220t00D e111S6K66Sc,,,    ea888S,r,n, c  5v55a 111wen   r    .owrokrfkloflwow. .M Moorreeoovveerr,,  tthhee  ddaatata v vereirfiicfaictiaotnio pnr opcreoscs ewssa sw garse agtlrye aatltleyr eadlt, earse din,  tahse  i99n9   oo otfff h  111e666     ODIKnOI DnSthc Kteah nSreco  prauonrtuo ipnctrieeonscdese  adsttshaa etthma fe im aefnilaedaln dgwa ewgomeormerkkneeetrnr p ttto ropooorckkoe  cttshheses,e s ass,og oalgelger gre errgseeapsgtpoainootsinnoibsoniilbf iotiryflei  rctoeyofc  rvoodifres dvwrsieie nrqweguq iainurnegidrd  eavmdne drmai fnvyaueinnarugilfa yddlia adnttaaga,t  edawn aehttnrialyte,r  wiyn ihtnoitleo   In the routine data management process, aggregation of records required manual data entry into  aisanp s rtpehiranIeIendn  atr s hdttohhhesu eeehrte  oreirtnouoe,euttua,i  tntndaiienndna eedtdda  ad d ditmagaait tigamataii n tzmmaiaaznagataainnetgoimaaeongmgneeoe enmomfntf toe e ponpnnrnerttoo e ppcrc reerereocsoscssocco  erettrdshshdssiis,t ,s   toaa roorgegoeskggkpsrrp 2oee2nogg mmsnaaistibtiniiiniobol.i.nni tlTTy  ioo thhwyffii   rssrawe esddcc aisoiogshgrr iaiddtstrihsisezz  dararaere ttbeqiqioeoduuntn wii brrpepeeerdedtronwo  mcm cefeeiaeasesnsnnlsd u uw wfawaiaellao s lddsd rraka retewtedaard  ou eeaurcnnnkcetdteerdr dyry  t  ioaitnnon 1ttdoo0     a spreadsheet, and digitization of one record took 2 min. This digitization process was reduced to 10  10Dasas   sMisnippn ODrrteethM.haa eIdedOn OssO .ht hDIeDenereKe mKttte ,,Sr s Saam cconnaasfddn  not   idfdwwm itigigooemiirr,tt kekiivz,zffl elavaoortetiwiwirfooiif..ncni acMM  oaotitffoooi  oorronneenn o ooeoevvf f r r eeppeerrcaac,,o poptthrerhedrdere   b  dtbtdaooaasaootsetakkeda d   v22rv e e rmemcreroicifiiirnfondci.r.cas  dTaT ttstihohoi ooitinsoskn  o dpd 2pkir0iggor  2soiic t0tcpeii zezsesassar ps ttrwi ieeowocranno asr  rpspegd crrg roofoerocrcaerdeeta tsls thyfssloey  waw r Dl attaaMlehsstree  Oerr redeDe, dd,dwMuua,h sccOaieelise,dnd   iw  ntttohoh t e i1h1l0e0e     OssiDOsn   iiKiD nnnth  K Sitttenhhch   SaOeteehcn   DOeaOOp nOKDDDr pDo KKKScrKc  oe SSaS scSccncsecaaa astpnnnhsnr     etwwopwhficrooeooeer rcrsflkkekdsiesf ffilsllwltood o itwtwow o wtr.oo..k   ooMkMMerk rk3 ooo3t errmo rmreeeo ooitoiknonvvv o teepephkrrreee,, ,r r t t t thsrhrhhoeeeeeecl ce  os odddorrrdaalaedet tstfaa  aporf   oreovvv srtneeehp rtsrrehoiiiif fbffeniiiiicc ceslfaaialiitdbtettyiii lioowodloinnnt ofwy   rppvp koorirererofoor kw.ccvc eTeeeiirsnehss.sswse gT    wwiwiahnnnaceaagrds ssi e  an agvggncsrerrerdeereedai aaavf ttctyslelloyyieyrnn d i asagfa uylcllttdtmioeeneanrrprgteeesat dduidd,o,,m,wan   aata pohasssf , it   iliiiwnenon n  ihtttn hhiholeeeef      thOtOOieinmDDDr oteKKKhtu iei  m tSnSS icnrc ectoaa aehinunnned   t  tppipavhnrtrereaeooor v ciccmdfeeeeirascsasistsafsnai   tcttat ihahhomgteeenieo a  m fffsnniiiete eeaselllntgddpdete   p wowwmp ofroo oefto rrhnrtckkhketeeee e spO rrrOs   rttDtDtoooohcKooKoiekskk  sSS   rsttccteh hhaatseneehnp    i sisosissooso n b llrlbseeeeeie c  sbrrracpeeieualssosiuspptpneysoo oseonnwin fbo ssstaifiihilb bsbitetihii syllaliheii pttt wyayyapr  p  aooewopsdfffo     vvsvwrbhikiieeeoefatwlwwrwrokewiiienfdnnl egogi gnnb w  avaaefinon ntiewlddndvl dv ie vvnveoeegwenlrr vri ioiidfifffrnyiyeyekngiiilnenntd iirgggfd i a w cedddnanaoaatditttrtoiaaakfDn,,i,e  c  wMwwra thhaOhioiniil.llndeee      in thaen dr oruecttiinfiec adtiaotna  omf tahnea mgeismreecnotg npirzoedce mssa cthhiinse r‐reesapdoanbsleib dilaittay  fiweladss  sbhya mreodv inbge tpwoeinetn‐b fyi‐epldoi nwt.o Trhkee r and  InaiDinntneM  dtrthhm Oreees.   crrIotonoifufu itttcteiiiamrnnmteeieo  s,ddn voa aeoftt arafti  i fitmmmhceeaaa ,ntnm ivaaoeiggnsreeriomemfifcceeopanngtatitnp o pipenzrrr eooodbccfa  eepmsssaessa pd cttehhhrrieii snsbc  eoarr‐eresrdseseppdsaoo dtrnonaeosscbiikolbberii2 dlldi0isttay ystt  oapwwo efakaires s2rl  des0shchs so aab rrprydeee ddrmf   orbboreeevctttohiwwnregdeeDe e pfnnMoo  rffi Oiniteeht,ll‐dedwb   yDwhw‐MipooleorrOkikinne,e trwr.t   haTahennhilddee     DMOO.D IKn  Stecarmn psr oofc etsims per,o vveidreifdic aa tsiiognni foicf apnat ptiemr eb raesdeudc trieocno frodr st hteo odkat a2 0tr sa npseferr r aencdo radg gfroerg tahtieo nD sMteOps,;  while  OODDDinMDMK thKOOSe c.S.  aO IcInnnaD n pttKe erprr oSmrmcocesascs  neooss ffpsi  t ttrpiitommorcooeeevks,,  isvvd3 ieeetmrr dtiifof iiainocc aksapitt gei3ioor nmnnirf  eioionccffoa   pppnredaatr ppt fireoemerrrc  ebbot haarredessede fifdduoe  crrlrd teetichcowooenrr o fddfiroesskr l etd ttoorh .owoeTkk od h 2r2aek00te i asnsr   .tcpp rTreaeehrnra  esrsr efeeienccdrooc crarredodnan  dffssoo eaurrdg m t tgchhproeeetn  igDDsoauMnMtmiooOOpnf,,t  t swiwiomtehnhpei iollseef;     in thheo OwDevKer S, ctaimn ep rroecqeusirse idt  tfooork t 3h em viner pifeicra rtieocno radn dfo rf otrhme  ffieillldin gw owrekreer .i Tnchree aisnecdr eaass ecdo mcopnasruedm ptot ion of  inhitinnitomh  twtehheceev eoiv  nerOOerr etriDDhfis, pKeKcto aiv  nSmtSeidccoreaiainn nfngir sc  peptpaeqrrtrpooiuoocccionereefess sssdsstteh   eisitetp f o ott Ofooor to foDh kktteKhh   33reeo S   mmOucvtaiDiiennnnrK e ippif s dieeScabrrcat  earatrciene omacc niouoas rsrn bddeaae ng offceodofamrr ut  htfetshohneereet a mo  (ffpTfii  epeatlfhbldidwlele l  wiw oa2npr,oog kFprr flikk gwwoeeurwreor..re r TTe ki6n hhf)vl.iee ono  iwiclnnrv ceciirnraneesvgaaeossdildeev ddeia nn cscgto oi fincnidocssuaemuntmmiptoipapnfrittcieiaooadnntn id o otoonff     time in the verification step of the ODK Scan is because of the app workflow involving identification  retcatciiontmmirdfiree ec  riasiennptci  ototthihnnfeeidc o vivafnetetgirrohi ifpnfeiir ccmooaafctti iisetoorhsnensec e  ssosmtt geeoipnpsf  ri toozehfefc e dott hhrgmoeen u OiaOztciDeDhndKieKn  m d eSS-acacrtaaceanhna m id iinssaae  bbbn‐lreeaeeccgaadaeduuamasstaebee  lnofioetffe   d (lttdThhasaetea b baa lfyppeie ppm2l  d,ww oFsvoo ibgirrnykukg frfmlleopo wo6wo)vi . ni iinnnt-vgvbo oypll-vvopiiinonngigtn‐  biitdd.yeeT‐npnhtoteiiiffnOiicctaDa. ttTKiioohnne    and rectification of the misrecognized machine‐readable data fields by moving point‐by‐point. The  ScaOaannnDddpK  rrr eoeScccctteaiisffnisicc ppaatrrtioiooovcnnei d sooseff d p Ttthrahaobeesvl  iemmgi d2ni.ie sisIfidrmreec pacacao onscggtti gnontinfiim zzOifeeeDiddcKra  em mnSdctaua atcnccihm htoiiionnen nv eera‐‐ferrrodieeorauautddsch tapaeibrobdolnlceaee  t fsdadoseaartsr tt taaaoh fn f efDsii eefdaelltaddart ssMaa  n bbatdnryyaa  agmnmgesmgoofervveerniig nnta.ag gnt  dippo oonaiignsngttter‐‐pbebgysy;a‐‐phptioooowiinnn etts.v.t  eTeTprhh,see;    ODK Scan process provided a significant time reduction for the data transfer and aggregation steps;  timOOhoDeDwKKree  qvSScuecarai,rnn e  tpdpimrrfoooeccr eesrstTessh q aeppburrlviooerev ve2ridi.idd fiI emecfddoap  traaai co  ststnh iioggefann  OniivffdDiieccKrRafaio onfnSiurtctcm t aatitninitmmie fioo Delennl  a i rrvnteeaaagd dnriuudowccu tetsfiir ooopernnrmo i fnfco oecrrfsr i setltehlhaisense oe  gddfd  aDawttaaaaste  attrcr reMaoa nnmaisnnTspffaiceemagrrreree  aeama Cndnseoeddnndt  otsaa .u ggamcggsoer rrdeerc ggeionasam pttiipooonannrd  sesittdnee gpptsso;;    prhhhcooooocrwwwerseeeessvvvepeseeorrroD,,n,   fadtttttiiaiihmmmn Megeeera    ponrrruareeegotqqqeicnmuueueisieirrrsndeeeetdda dsPt    roaoffffocomo etrrrhs  a sentett hhshar eeoeg   u eMvvvmtieaeenenrrRreiniaiffo fgitdiiucecc(amataaTittttnaeaiiinooeb omt nnlnD eP  a ra2anaoat,ncananeF dgddsise  g  mffufooorerrOrenmmmD6t   ) K(.fTff iiSilallclllbiiainnnnlegg gP   2 ro,www cFeeeeisrgrrseee u   riieinRnn 6ceccdr)rr.eeee Tasaaiisssgmeeendddeed    C aaPaossrsno   scccceuooosmmsmm epppdaaa irrrneee ddd   tttooo    cccooorrrrrreeeDssspppaootoannn MdddFiiiaonnnnrgggma   pgp pFerrirlmoooliccnceeeegnss st(sssp Peeeesrssro    fooocoffefr   msttthshh)e eees    rrr ooouuMutttiiainnnneeea   g1ddd emaaamtttiaanae    nm2mm7t  aaaPs nnnr oaaacgggeeeesmmms eeennnOttt   (((D2TTT Kmaaabbb iSnlllee ce5 a  7222n ,s,,     PFFFriiigoggcuuuerrrseees    666)))...   Redes igned Process  DDaFatatoa DrMA mCagtaoTa gFn lrrCliaaeelongglcTilsaetnlfatieTTmTTegiobcoraa tnaa len(ib(bbbeop cn ((llnollcep2eteee mo r.(e  P   2c2m22rfIpor. .om.. r ol  mpIIereIImcmpmmmtclpeeeoapl)pstpp)re ce sdaataat)eec)ccc o )stttt   f    oooo Offff    ODOOODKDDDRKKRKKoS uc o  1SSSS3a13t uccm––cc–in2anat24aa4 inmnoi  nne nddnd n   ioeaaoD oona2yy nvnnny D7ass a   s   vtvvvsar a  aa iaat ora rrriui iioooosuuuupssss r   pppporrcrrooooeOccscceesDees3essss–Kssss32ee41ee3 o  ms s0ssSm–df     o4oiscaooDni f ayfnffd     nasDD DDa 5t aaPya7aatts rttMasaaa o     cMMaMMensaaaaasnnnng aaaaeggggmeeeeTTmemmminimmeeeetnn.nneetttt ..C  ..C     oonnss uu mmeedd iinn  DatDaaMtaa TMnVraaaegnrneisfamfigecaeret nm(ioctoenPn m(rptop ePcrlere rosteescce)eo ssrsde) s   MManaanRRRgaooeog1uumue–tttm22ieiinn0 nned eesetna    DPDtyD rsPaaao rttt caoaae  c sesss  OODDKK3  mSS3cci naam nni nPP rroocceessss  RTRTTeiieimmdmdeeeeess    CiCCiggooonnnnneesssdd uuu PmmPmrreoeeoddcdce  e iiissnnnss     DDaaAttFagao  gMMrrmeaag nnFaaaitlgigloieennmmg ( eep(npnetetr   rPPr efrroocoroccmeredss)ss)  ee ss     MMMaaannnaaagg1gee em2mmm mieenennn i2nttt 7   PPP srrr ooo ccceeessssss    OODDKK2   SSmcc1aia0nnn  s 5  PP7rr oso cceessss   RRReeedddeeesssiiigggnnneeeddd     PPPrrroooccceeessssss    FoDrVFmFFaToeotoahrrrFri mmemiCfli lrc  o iFFeFanlidiltigllelillelociii(snnntnpiiggg goe(  n pnr(((ppep ef(oderece rrr orrd m  efmffaooco)troprrammmrl edm))t))  e  a) n agemen1t1 11mp3   mmrm–ion42iiicn 0nnd2e    s7s2a22 s7y77s   ss sssa     v   ed resources2222 3 b  mmmm–3y4 ii iminen nndl   ii55a55nm77y77   is sssns    ating tasks that dem      anded  DtaimtDDaDeaaCa atttoaaTan l  drCClCea oomconltllslilolefoeeenccncrtett iii(y(ooocc.nnon oI  mn m(((cc ctpopoohlmmlmeee ttprppeeoll)l)eeue tttteeei)n))   e data ma3n–3313a4––––g4424de    ddamddyaaaaeysyyynsssst      p   rocess, paper333‐3b––––3a4444 s  m dddedaadaianyyy yr sssse   cords were sent to th e    office  foTrhT Ardeagi nTrgTTgesirrrrtfdaaaieeznnnegrassssa(tfffitcieeegioororrnn m  n((( eccc vpo(doopilmmma eed trppplea ol)rllteeceeatattce eelom) )) r c dao)nu ra igere mseervnit1c ep–111.2––r –22Ho22d   m cddaodeyawaiasnysyyse sss v  s e  a rv iend t rhees roeudrecseig33s33 n 1  mmbmm0eyd iiiisnn nn  e p  lrimociensas,t imngan tuaaslk esn tthrya     to df etmhea nded  timeAr eagcngAoAAVdrrgeggd emggggrsarir rfoeaeteiiggtcngo aaaaenttthttyiiii(oeooo.p  nnnnIoen    r(f(((pfp pptrieheceeerercrre    o rr rrirreeseeodc ccccouo)ooorrrtrmdddidn))))p   e   l   edtaeltya  meliamni2an222mga  2 mmtem0iemn idiisnnn  e  a nntd p droatcae siss ,s penatp teo3r11 11‐ t0m0b00h   assesisn s   se edr vreerc oerledcstr woneircea lslye nt    th troou thghe  office  for diVniegteriVViVrtfinieeeczerrraaitiiftf fticiiiicoccooaananntttini(i ooopevnnneci  tr a(((ipppv releeieotcrrry oc  rrr.are eedTlccc )ohoocrerrodd dur)))e  r d  i eers igsnerevdi cdea.t2 a02H22 0m00so   sssaw   neavgeemr einn t tphreo creesds e333o3s  f mmmimfgeiiirninnnnee  d d f lpexroibcielistys,  tom saunpuearvl    iesonrtsr yin  of the  recorreTdvhsi eeaw rt ientdhgee ds ioiggfinftiaeclde d  diasta act aao nmmdp aenlneaatbgelleeymd e etlhnimetm pin rtoao tcpeerdso sva isndadev  fededea dtrabe asicosku  siremcnemts  etbdoyi a tehteleilmy s. ien ravteirn ge lteacstkros nthicaatl ldye tmharonudgehd   The redesigned data management process saved resources by eliminating tasks that demanded  itnimteer nTTaehnhtede c   rrmoeenddoneneesseciiyggti.nnv Ieenitdd yt  h.dd eTaa thrtaaoe  u mmrteiaandnneeaa sdggigaeetmnmae eemdnn atdt  npapatrragoo eccmmeessaessn n sastaag pvvereemoddce  rerneestsss op,o puuroarrccpceeeesssr s ‐bb boyyaf  sfeeeellirdimme rdiien ncfaaloettrixidnnibgsg i  wtltiaatessyrkk etss os  tte hhsnuaattpt  t dodere etvmhmiesaa oonnrfddsf ieeicdnde     time and money. In the routine data management process, paper‐based records were sent to the office  rttfieiommvr ieede  awaignniiddnti g mzm adootiinngoeeintyya .v.l  I Iidnnaa   ttlthhoaeec a  arrnolo duuc otteiiunnnreaei b eddlraea tdstaae   trmmhveaiacmnneaa.  tggoHee mpomwreeonnevvtti  dpeprerr o oifncceeee stsdhssb,,e  app rcaaekppd eieemrrs‐‐imbbgaanessdeeeddida   rtrpeeerlccoyooc.r redd ssss  ,ww meerraeen  ssueeannlt t e ttnoo t ttrhhyee   ooofff fftiihcceee    for digitization via local courier service. However in the redesigned process, manual entry of the  frfooercr  oddriidggsii ttiaizzta attthiiooenn o  vfvfiiiaac  ell ooicsca acll o ccmoouuprrleiieetrer  lsysee rervvliiimcceei..n  HaHtoeodww eeavvneedrr   diinna  tttahh eies   rrseeeddneetss itigogn nteehdde   ppserrorovcceeerss sse,,  lmemcaatrnnouunaailcl  aeelnnlyttr rytyh  orooff  uttghhhee    records at the office is completely eliminated and data is sent to the server electronically through  rirneectceoorrrnddesst   aactto  ntthhneee  cootffiffviicicteey  .ii ssT  chcooemm reppdlleeettseeigllyyn  eeedllii mmdaiinntaaa ttmeedda  naaanngdde  mddaaettnaat   iipss r ssoeecnnetts  stt ooo  fttfhheeer e ssdee rrfvvleeexrri  beeillleeictcyttrr otoonn iisccuaaplllleyyr  vtthhisrroooruusg gihhn    internet connectivity. The redesigned data management process offered flexibility to supervisors in  irinnettveeirrennweeittn  ccgoo ndnnnigeeiccttatiilvv diittayyt..a  TT ahhneed  rr eeenddaeebssiilggendne etddh  eddmaat ttaao  m mparaonnvaaiggdeeemm feeenendtt  bpparrcookcc eeismsss m ooffeffdeeriraeetdde  lffyllee. xx iibbiilliittyy  ttoo  ssuuppeerrvviissoorrss  iinn   reviewing digital data and enabled them to provide feedback immediately.   rreevviieewwiinngg  ddiiggiittaall  ddaattaa  aanndd  eennaabblleedd  tthheemm  ttoo  pprroovviiddee  ffeeeeddbbaacckk  iimmmmeeddiiaatteellyy..       FigurFeig6u.rRe e6d. Reseidgenseigdnedda tdaatma amnaangaegmemenenttp prroocceessss iinn OODDKK SSccaann ppapaepre‐tro-‐tdoi-gdiitgali tsaylssteymst.e m. Theredesigneddatamanagementprocesssavedresourcesbyeliminatingtasksthatdemanded timeandmoney. Intheroutinedatamanagementprocess,paper-basedrecordsweresenttotheoffice   for digitization via local courier service. However in the redesigned process, manual entry of the Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system.      Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system  .  Figure 6. Redesigned data management process in ODK Scan paper‐to‐digital system.  FFiigguurree  66..  RReeddeessiiggnneedd  ddaattaa  mmaannaaggeemmeenntt  pprroocceessss  iinn  OODDKK  SSccaann  ppaappeerr‐‐ttoo‐‐ddiiggiittaall  ssyysstteemm.. FutureInternet2016,8,51 10of17 records at the office is completely eliminated and data is sent to the server electronically through internetconnectivity. Theredesigneddatamanagementprocessofferedflexibilitytosupervisorsin reviewingdigitaldataandenabledthemtoprovidefeedbackimmediately. 3.3. FeedbackonAppUsabilityandWorkflowModifications Thefieldworkerslearnedquicklyhowtousethesmartphonesandswiftlybecamecomfortable withtheapplication’sworkflow: mostofthemwerealreadyusingsmartphonesintheirpersonallives. Nevertheless,theimplementationofODKScanwasguidedbytheprinciplesofminimaldisturbance incurrentworkflowandgeneratingvalueforuserstoimprovethechancesofadoption. Face-to-face interviewswereconductedindependentlywitheachofthefieldworkersattheendoftheintervention period. Theseinterviewsweresupportedbyasemi-structuredquestionnaire,whichincludedhintsto guidetheinterviewertoensurethecompletenessofthefeedback. Theinterviewswereconductedby aresearchteammemberinthelanguageoftheenrolledfieldworkers(Urdu). Allinterviewswere digitallyrecorded,transcribedinUrduandthentranslatedintoEnglish. Theinterviewrecordings andthecorrespondingtranslationswerethencheckedbyanotherteammemberforconsistencyand completeness. Data is segregated based on the various steps of the app workflow for the ease of analysisandpresentation. 3.3.1. FormFilling Inbothroutineandredesigneddatamanagementprocessesusersarerequiredtofillpaper-based form. Theformdesignsusedinroutineandredesignedprocessesarecompletelydifferent,butdata fieldsarecommonbetweenthetwodesigns. Furthermore,theScan-compatibleTB03formallowed morestructuredandfewerhandwrittentextfields,whereasthestandardTB03formincludedhand written data fields only. The ODK Scan process begins with filling in the ODK Scan-compatible form. This includes marking the correct bubble for the fill-in bubble fields, linking guide dots for thestructurednumberfieldsandwritinginthetextfields. Fieldtestresultsshowedthatthisstep took more time as compared to the corresponding step of the routine data management process. Moreover, in terms of learning to fill in the new form, all of the field workers adapted to the formwellandfounditeasytolocatethedesiredfield. Nonetheless,almostallfieldworkerswere unhappy about the time-consuming aspect of the form filling for the ODK Scan process (Table 3), while at the same time they acknowledged the benefit of using number boxes and fill-in bubbles. OneDFSsaid,“Thisexperiencewasverygood. Thisisexcellent. Especiallydotsandbubbleshave eliminatedanychanceofthedatamanagementofficercontactingusforclarificationortoaddressany readabilityissue.” Interestingly,oneDFShighlightedadifferentaspectofthis,saying “[I do not like] filling the dots on the form as they are time consuming. Also, filling the dots embarrasses me whenever I do that in the presence of others. It seems like I am a kid and practicinghandwriting.” Table3.Self-administeredquestionnaireitems. LearningtoFill Locating Easeof Easeof FieldWorkers(FW) NewForm DesiredField WritingNumbers FillingBubbles 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 FW1 FW2 FW3 FW4 1correspondstoveryeasyand5correspondstodifficult.

Description:
mHealth Coordinator, Mercy Corps, Pak Palace, Murree Road, Rawal Computer Science and Engineering, University of Washington, Seattle, WA 98101, USA; programs mostly maintain and manage paper-based patient recording and reporting systems [6]. resource for front line workers [10].
See more

The list of books you might like

Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
DMCA & Copyright: Dear all, most of the website is community built, users are uploading hundred of books everyday, which makes really hard for us to identify copyrighted material, please contact us if you want any material removed.
Copyright @ PDFdrive.to, 2022 | We love our users