Low Computational-Cost Footprint Deformities Diagnosis Sensor through Angles, Dimensions

17 Pages·2017·6.19 MB·English

sensors
Article
Low Computational-Cost Footprint Deformities
Diagnosis Sensor through Angles, Dimensions
Analysis and Image Processing Techniques
J.RodolfoMaestre-Rendon1,2,TomasA.Rivera-Roman1 ID,JuanM.Sierra-Hernandez3,
IvanCruz-Aceves4 ID,LuisM.Contreras-Medina5,CarlosDuarte-Galvan6 ID and
ArturoA.Fernandez-Jaramillo1,*
1 UnidadAcadémicadeIngenieríaBiomédica,UniversidadPolitécnicadeSinaloa,CarreteraMunicipalLibre
MazatlánHigueraskm3,Col.GenaroEstrada,MazatlánSin.82199,Mexico;
[email protected](J.R.M.-R.);[email protected](T.A.R.-R.)
2 CenterforBiomedicalTechnology,PolythecnicUniversityofMadrid,CampusMontegancedo,
PozuelodeAlarcón,Madrid28223,Spain
3 DepartamentodeIngenieríaElectrónica,DivisióndeIngenierías,UniversidaddeGuanajuato,
CarreteraSalamanca-ValledeSantiagokm3.5+1.8,ComunidaddePaloBlanco,
SalamancaGto.C.P.36885,Mexico;[email protected]
4 CONACYT,CentrodeInvestigaciónenMatemáticas(CIMAT),A.C.,JaliscoS/N,Col.Valenciana,
GuanajuatoGto.36000,Mexico;[email protected]
5 CAIngenieríadeBiosistemas,DivisióndeInvestigaciónyPosgrado,FacultaddeIngeniería,
UniversidadAutónomadeQuerétaro,CerrodelascampanasS/N,
SantiagodeQuerétaroQro.76010,Mexico;[email protected]
6 FacultaddeCienciasFísico-Matemáticas,UniversidadAutónomadeSinaloa,Av.DelasAméricasyBlvd.
Universitarios,Cd.Universitaria,CuliacánSin.80000,Mexico;[email protected]
* Correspondence:[email protected];Tel.:[phone]
Received:19October2017;Accepted:14November2017;Published:22November2017
Abstract: Manualmeasurementsoffootanthropometrycanleadtoerrorssincethistaskinvolvesthe
experienceofthespecialistwhoperformsthem,resultingindifferentsubjectivemeasuresfromthe
samefootprint. Moreover,someofthediagnosesthataregiventoclassifyafootprintdeformityare
basedonaqualitativeinterpretationbythephysician;thereisnoquantitativeinterpretationofthe
footprint. Theimportanceofprovidingacorrectandaccuratediagnosisliesintheneedtoensurethat
anappropriatetreatmentisprovidedfortheimprovementofthepatientwithoutriskinghisorher
health. Therefore,thisarticlepresentsasmartsensorthatintegratesthecaptureofthefootprint,alow
computational-costanalysisoftheimageandtheinterpretationoftheresultsthroughaquantitative
evaluation. Thesmartsensorimplementedrequiredtheuseofacamera(LogitechC920)connected
to a Raspberry Pi 3, where a graphical interface was made for the capture and processing of the
image,anditwasadaptedtoapodoscopeconventionallyusedbyspecialistssuchasorthopedist,
physiotherapistsandpodiatrists. Thefootprintdiagnosissmartsensor(FPDSS)hasproventobe
robusttodifferenttypesofdeformity,precise,sensitiveandcorrelatedin0.99withthemeasurements
fromthedigitalizedimageoftheinkmat.
Keywords: embedded system; footprint measurements; Staheli arch index; Clarke’s angle;
Smirak-Chippauxindex;biomedicalimageprocessing
1. Introduction
Itiswellknownthataccurateandquantitativemeasurementsarecriticalwhenaclinicaldiagnosis
isgiven. Furthermore, manualmeasurementsoftheanthropometryofthefootcanleadtowrong
Sensors2017,17,2700;doi:10.3390/s17112700 www.mdpi.com/journal/sensors
Sensors2017,17,2700 2of17
estimations because of the subjectivity involved in this task or the training and experience of the
personresponsibleforthemeasurements. Thediagnosisoffootdeformitiesinindividualsrequires
multiplemeasurementsfromthefootprinttocalculateasetofparametersthatdeterminecharacteristics
presentedinthearch[1]. Studieshaveshownaninconsistentperformanceofmeasurementsofthe
footwithtraditionalinstruments,likeacaliperoraninkmat[2–4]. Whentheorthopedistgivesan
incorrectdiagnosis,theappropriatetreatmentwillnotbegivenintimeand,consequently,itwould
affectthehealthofthepatient. Inordertobeabletotracktheimprovementofthepatientovertime
andchecktheeffectivenessoftheprescribedtreatment,itisnecessarythatmeasuresbequantitative,
consistentandpreciseeverytime. Oneofthemostcommondeformitiesinthefootistheflatfootor
pesplanus. Flatfootisaconditionwherethemediallongitudinalarchofthefootcollapsescausing
partialorcompletecontactwiththegroundthatwillcausesevereproblemsinthepatient[5].
Conventional methods used in the clinical environment involve the use of a caliper by the
orthopedist to estimate the required measurements of the foot. It is rudimentary and exhaustive
to manually take all the measures and the variability, produced by the lack of precision which
willleadtoerrorsinthecalculatedparameters.Thereareseveralmethodologiesproposedforthe
computer-aided diagnosis of foot deformities. For example, Navarro et al. [6] designed a special
soletemplatewithpressuresensorstoevaluatethedistributionoftheplantarpressureindifferent
key parts. It is a good approach; however, it is necessary to manufacture different sizes of sole
templatestoadaptthepositionofthesensorstospecificfootsizesinordertoobtainprecisemeasures
in the appropriate areas. Hamzaetal. in 2015 [7] used an ultrasonic sensor that passes under the
foot and acquires the height of the medial longitudinal arch. Nonetheless, the use of ultrasonic
sensorsto measure thefootprintin astraight linewillignorethemeasurementsof differentareas,
not present in the arch, which could be substantial when a diagnosis is given. On the other hand,
Thereareseveralcomputationaltechniquesthatareresponsibleforreconstructionofthefootprint;
forexample, Guerrero-Turrubiatesetal.[8]proposedtheuseofaparaboladetectionalgorithmin
the areas of the arch and the heel of the foot, also the finite element method are widely used to
reconstructthefootprint,eventhecompletefootorthepressurecausedbyitself[9–11].Bothproviding
excellentresultsbutwithahighcostintermsofcomputingresourcesandtime.Itisveryimportantto
considertherequiredresourceswhenimplementingthealgorithm,especiallywhenitisnecessaryto
generateanapplicationforembeddeddevicesordispensewithahigh-performanceandexpensive
computer.Arecentstudy[12]presentedasystemfortheanalysisofthefootarchbyusinganRGB-D
camerafor3Dimagereconstructionofthefootprint;suchsystemhasshowngoodreliabilityinthe
performedexperiments.However,themeasurementsofthefootprintandtherelationoftheforefoot
andthebarefootwiththemidfootareleftbehindsincetheyonlyfocusontheanalysisofthefootarch.
Consequently,thecompleteanalysisofthefootprintcannotbegivenandthiscouldpotentiallygenerate
aninsufficientdiagnosis.Currently,therearenoRBGcamera-basedsensorsthatautomaticallyanalysze
themeasurementsofthefootprint. Table1showsacomparisonbetweenthemethodsdescribedabove
andourcurrentapproach(FPDSS).
Sensors2017,17,2700 3of17
Table1.Comparisonofthemethodsusedbydifferentsensorstodiagnosefootdeformities.
Authors/Method ObjectiveoftheMethod TypeofSensor Accuracy
Detectionofflatfootusing
Navarroetal.[6] Pressuresensors N/A1
estimationtechniques
Flatfootdetectorusingthe
Hamzaetal.[7] Ultrasonicsensors 100%in20subjects
heightofthearch
Detectionofparabolas
Guerrero-Turrubiatesetal.[8] Footprintscanner N/A1
dimensionsinthefootprint
3Dimagereconstructionof
Chunetal.[12] RGB-Dcamera ~97.32%in11subjects2
thefootprint
Detectionofdeformitiesin
FPDSS thefootprintthrough RGBcamerawithreference 99.38%in40subjects2
imageprocessing
1Accuracynotspecifiedinarticle.2Meanoftheaccuracyinthethreeparameterscalculatedbythesensor.
Sensors 2017, 17, 2700 3 of 16
Therearesomedevicesinthemarketthatprovidedigitalimagesofthefootprintbutonlyathigh
pressuTrehearree aasr.eA sodmdiet iodnevailclye,s itini sthnee cmesasrakreyt fthoratt hperouvsiedre tdoigenittaelr imquaagnetsi toaft itvhee mfoeoatspurirnetm beuntt sonolfyt hate
fohoitg.hT hperesesspurroeb aleremass. dAodndoittiaolnloawllyt, hite iasu nteocmesastaicryc aflocru tlhaeti ounseorf tpoa ernatmere tqeurasnretiqtautiirveed mfoeratshueredmiaegnntso soifs
the foot. These problems do not allow the automatic calculation of parameters required for the
of deformities in the footprint. In Figure 1, the conventional devices used in the diagnosis by the
diagnosis of deformities in the footprint. In Figure 1, the conventional devices used in the diagnosis
orthopedistcanbeobserved. Thedataobtainedfromthedigitalcaliperandtheinkmatwillalways
by the orthopedist can be observed. The data obtained from the digital caliper and the ink mat will
resultinmeasurementsthataresensitivetothetrainingoftheorthopedistbecausetheareawhere
always result in measurements that are sensitive to the training of the orthopedist because the area
themeasurementmustbemademayvaryfromonepatienttoanother[2]. Computerizedpodoscope
where the measurement must be made may vary from one patient to another [2]. Computerized
(PlantoscopioComputarizado,Puebla,Mexico)isanopticaldevicewithwhichitispossibletoscan
podoscope (Plantoscopio Computarizado, Puebla, Mexico) is an optical device with which it is
and visualize the footprint without any analysis that determines the condition present in the foot.
possible to scan and visualize the footprint without any analysis that determines the condition
In the same way, ArcoScan (CiPar Ingeniería, Paraná, Argentina) is a podoscope that allows you
present in the foot. In the same way, ArcoScan (CiPar Ingeniería, Paraná, Argentina) is a podoscope
to digitalize and store the footprint but, with the difference to the previous one, it also allows the
that allows you to digitalize and store the footprint but, with the difference to the previous one, it
estimationofareaswithhigherpressurebyusingtheintensitylevelsofthepixelsintheregionof
also allows the estimation of areas with higher pressure by using the intensity levels of the pixels in
thefoot. However,thesedevicesdonotincludeaqualitativeorquantitativeanalysisofthefoot,like
the region of the foot. However, these devices do not include a qualitative or quantitative analysis of
measurements and angles of the footprint, and they will always require the interpretation by the
the foot, like measurements and angles of the footprint, and they will always require the
orthopedist in order to give a diagnosis. Having measurements on a real scale and the automatic
interpretation by the orthopedist in order to give a diagnosis. Having measurements on a real scale
calculation of parameters that indicate deformities in the foot is very important to provide more
and the automatic calculation of parameters that indicate deformities in the foot is very important to
efficientdiagnosis. Itisimportanttomentionthatfootdeformitiesshouldbetreatedasquicklyand
provide more efficient diagnosis. It is important to mention that foot deformities should be treated
efficientlyaspossibletoreducetheeffectstheymaycauseinotherareasofthebody[13–15]. Whenthe
as quickly and efficiently as possible to reduce the effects they may cause in other areas of the body
deformityiscorrectlydiagnosed,anappropriatetreatmentisgiventothepatient.
[13–15]. When the deformity is correctly diagnosed, an appropriate treatment is given to the patient.

(a) (b) (c)
Figure 1. Traditional devices used for the measurement of the foot: (a) Digital caliper; (b) Ink mat; (c)
Figure1. Traditionaldevicesusedforthemeasurementofthefoot: (a)Digitalcaliper;(b)Inkmat;
Digital podoscope (modified from [3]).
(c)Digitalpodoscope(modifiedfrom[3]).
However, this process can be automatized using a digital footprint analysis smart sensor, which
However, this process can be automatized using a digital footprint analysis smart sensor,
would considerably take out all the disadvantages of manual measurement while computing the
whichwouldconsiderablytakeoutallthedisadvantagesofmanualmeasurementwhilecomputingthe
parameters needed for the correct diagnosis. The objective of this article is to introduce a sensor that
integrates the capture of the plantar footprint, the analysis of the image and the interpretation of the
results through a quantitative evaluation. The advantage over conventional methods lies in the ability
to automatically detect footprint measurements and the angle of the longitudinal arch of the foot in
a real scale quickly and with minimal computational resources and, in the same way, the calculation
of three essential parameters to fully describe the footprint quantitatively: Staheli index, Clarke’s
angle and Smirak-Chippaux Index. Consequently, it provides an accurate interpretation of the results
for both, the diagnosis and the orthopedist. Finally, the calculated parameters from the patient’s
footprint are saved and plotted to graphically observe the improvement over time to ensure that the
treatment given by the orthopedist is right for the condition, so the patient’s health is not to put at
risk and the recovery time is reduced. A smart sensor is presented for the automatic quantitative
interpretation and analysis of footprint images resulting in a computer-aided diagnosis of deformities
in the foot.
Sensors2017,17,2700 4of17
parametersneededforthecorrectdiagnosis. Theobjectiveofthisarticleistointroduceasensorthat
integratesthecaptureoftheplantarfootprint,theanalysisoftheimageandtheinterpretationofthe
resultsthroughaquantitativeevaluation.Theadvantageoverconventionalmethodsliesintheability
toautomaticallydetectfootprintmeasurementsandtheangleofthelongitudinalarchofthefootina
realscalequicklyandwithminimalcomputationalresourcesand,inthesameway,thecalculationof
threeessentialparameterstofullydescribethefootprintquantitatively: Staheliindex,Clarke’sangle
andSmirak-ChippauxIndex. Consequently,itprovidesanaccurateinterpretationoftheresultsfor
both,thediagnosisandtheorthopedist. Finally,thecalculatedparametersfromthepatient’sfootprint
aresavedandplottedtographicallyobservetheimprovementovertimetoensurethatthetreatment
givenbytheorthopedistisrightforthecondition,sothepatient’shealthisnottoputatriskandthe
recoverytimeisreduced. Asmartsensorispresentedfortheautomaticquantitativeinterpretation
aSnendsoarsn 2a0l1y7s, 1is7,o 2f70f0o o tprintimagesresultinginacomputer-aideddiagnosisofdeformitiesinthefo4o otf. 16
22.. MMaatteerriiaallss aanndd MMeetthhooddss
2.1. SmartSensorSetup
2.1. Smart Sensor Setup
The implementation of the smart sensor presented required the use of a modified regular
The implementation of the smart sensor presented required the use of a modified regular
podoscope,aswecanobserveinFigure2;aonesquaredcentimeterblackstickerasareferenceareawas
podoscope, as we can observe in Figure 2; a one squared centimeter black sticker as a reference area
placedonthetopofatemperedglasswithafrostedfilm[13]andagenericcamerawasembeddedunder
was placed on the top of a tempered glass with a frosted film [13] and a generic camera was
theglass,inthiscase,aLogitechHDProWebcamModelC920(Logitech,Lausanne,Switzerland)was
embedded under the glass, in this case, a Logitech HD Pro Webcam Model C920 (Logitech, Lausanne,
used. Thisallowedthecaptureofthefootprintinanimagefileforthecomputer. Theparameters
Switzerland) was used. This allowed the capture of the footprint in an image file for the computer.
forimagecaptureofthecameraweresetto40%brightness,50%contrast,60%sharpnessand60%
The parameters for image capture of the camera were set to 40% brightness, 50% contrast, 60%
ofthewhitebalance. AsshowninFigure3, thecamerawasconnectedtoanembeddedcomputer
sharpness and 60% of the white balance. As shown in Figure 3, the camera was connected to an
(RaspberryPi3,RaspberryPiFundation,Cambridge,UK)withtheRaspbianJessieOperativeSystem
embedded computer (Raspberry Pi 3, Raspberry Pi Fundation, Cambridge, UK) with the Raspbian
fortheanalysisoftheobtainedimagesthroughaninteractionoftheprocessingalgorithmandthe
Jessie Operative System for the analysis of the obtained images through an interaction of the
inputoftheuserinthegraphicalinterface.
processing algorithm and the input of the user in the graphical interface.
USB Cam era to
Raspberry Pi 3

Figure 2. Adaptation of the original podoscope placing a frosted film on the top and the camera
Figure 2. Adaptation of the original podoscope placing a frosted film on the top and the camera
connected to the Raspberry Pi 3 via USB. Using an USB camera, the Raspberry Pi 3 acquires an image
connectedtotheRaspberryPi3viaUSB.UsinganUSBcamera,theRaspberryPi3acquiresanimageof
of the patient’s footprints (left) and process it to display the results in the graphical user interface
thepatient’sfootprints(left)andprocessittodisplaytheresultsinthegraphicaluserinterface(right).
(right).
TThhee smsmaartrts esnesnosroarn danthde tihnev oilnvveodlpvreodc epsrsoccaenssb ecaapnp lbieed aipnpalnieydg einne raincyp ogdeonsecroipc epboydionstceogpraet inbgy
aincteagmraetriangt oa tcaakmeetrhae toi mtaakgee tshoe fimthaegefso ootfp trhine tfoaontdpruinsti nagndt huesisnogf ttwhea rseofftowraaren afloyrs aisn.aIlnystish.i sInw thaiys,
cwoanyv,e cnotniovneanltieoqnuailp emqueinptmceannt bcaeni mbep irmovperodvteod ptoro pvriodveidme omreoraec accucruarteatde idaigangonsotsictisc.s.F Figiguurere3 3 sshhoowwss
tthhee pprroocceessss tthhaatt tthhee ssmmaarrtt sseennssoorr ffoolllloowwss ttoo pprroovviiddee tthhee rreessuullttss ttoo tthheeu usseerr..

Figure 3. Block diagram of the smart sensor.
2.2. Footprint Parameters
Three parameters were used for the analysis and evaluation of the FPDSS images and obtained
data. Therefore, this section explains the importance of the metrics used and how they are obtained
in footprint images.
The Staheli arch index is used to describe the relation between the minimum width of the
midfoot and the maximum width of the hindfoot. This index has been one of the most studied
parameters for the description of footprint deformities and historically represents an important index
for the detection of flat foot [16]. In addition to the above, it is considered as an essential diagnostic
Sensors 2017, 17, 2700 4 of 16
2. Materials and Methods
2.1. Smart Sensor Setup
The implementation of the smart sensor presented required the use of a modified regular
podoscope, as we can observe in Figure 2; a one squared centimeter black sticker as a reference area
was placed on the top of a tempered glass with a frosted film [13] and a generic camera was
embedded under the glass, in this case, a Logitech HD Pro Webcam Model C920 (Logitech, Lausanne,
Switzerland) was used. This allowed the capture of the footprint in an image file for the computer.
The parameters for image capture of the camera were set to 40% brightness, 50% contrast, 60%
sharpness and 60% of the white balance. As shown in Figure 3, the camera was connected to an
embedded computer (Raspberry Pi 3, Raspberry Pi Fundation, Cambridge, UK) with the Raspbian
Jessie Operative System for the analysis of the obtained images through an interaction of the
processing algorithm and the input of the user in the graphical interface.
USB Cam era to
Raspberry Pi 3

Figure 2. Adaptation of the original podoscope placing a frosted film on the top and the camera
connected to the Raspberry Pi 3 via USB. Using an USB camera, the Raspberry Pi 3 acquires an image
of the patient’s footprints (left) and process it to display the results in the graphical user interface
(right).
The smart sensor and the involved process can be applied in any generic podoscope by
integrating a camera to take the images of the footprint and using the software for analysis. In this
Swenasyor, sc2o0n17v,e1n7,t2io70n0al equipment can be improved to provide more accurate diagnostics. Figure 3 sh5oofw1s7
the process that the smart sensor follows to provide the results to the user.

Figure 3. Block diagram of the smart sensor.
Figure3.Blockdiagramofthesmartsensor.
2.2. Footprint Parameters
2.2. FootprintParameters
Three parameters were used for the analysis and evaluation of the FPDSS images and obtained
ThreeparameterswereusedfortheanalysisandevaluationoftheFPDSSimagesandobtained
data. Therefore, this section explains the importance of the metrics used and how they are obtained
data. Therefore,thissectionexplainstheimportanceofthemetricsusedandhowtheyareobtainedin
in footprint images.
footprintimages.
The Staheli arch index is used to describe the relation between the minimum width of the
TheStaheliarchindexisusedtodescribetherelationbetweentheminimumwidthofthemidfoot
midfoot and the maximum width of the hindfoot. This index has been one of the most studied
andthemaximumwidthofthehindfoot. Thisindexhasbeenoneofthemoststudiedparametersfor
parameters for the description of footprint deformities and historically represents an important index
thedescriptionoffootprintdeformitiesandhistoricallyrepresentsanimportantindexforthedetection
for the detection of flat foot [16]. In addition to the above, it is considered as an essential diagnostic
offlatfoot[16]. Inadditiontotheabove,itisconsideredasanessentialdiagnosticmethodcompared
to the talar—first metatarsal angle, commonly used by specialists in the field for the detection of
d eformitiesinthefootprint[17].
Sensors 2017, 17, 2700 5 of 16
Clarke’s angle or arch angle describes the measure of the internal longitudinal arch.
method compared to the talar—first metatarsal angle, commonly used by specialists in the field for
Accordingto [18], thisparametermayhavebeenthefirst proposedforstudiesindeformitiesand
the detection of deformities in the footprint [17].
posture,asitisproportionaltothearchheight. Thisindexcomesfromtheanglebetweenalinethat
Clarke’s angle or arch angle describes the measure of the internal longitudinal arch. According
goesfromttoh [e18o]u, ttheirs ppaorianmteotefr tmhaeyf hoarveef oboeetnt othteh feirsot uptreopropsoedin fotro sftuthdieesh iinn ddeffooormtiatinesd aannd optohseturreli,n aes ict onnecting
theouterpiso pinrotpoofrttiohnealf otor ethfeo aortcha nhedigthht.e Tihnisn ienrdepxo cionmteosf frtohme tmhei danfogloet b.etween a line that goes from the
outer point of the forefoot to the outer point of the hindfoot and another line connecting the outer
Finally,theChippaux-Smirakindexistherelationbetweentheminimumwidthofthemidfoot
point of the forefoot and the inner point of the midfoot.
and the maximum width of the forefoot. When the arch of the foot is larger, this index decreases
Finally, the Chippaux-Smirak index is the relation between the minimum width of the midfoot
itsvalue[1a8n]d. tBhey muasxiinmgumdi wffiedrtehn otf pthaer faomrefeotoet.r sWthheant theen atrirche loyf dthees fcoroitb ise ltahrgeerf,o tohitsp irnidnetx, dthecerevaaselsu ietss obtained
during thevaalunea [l1y8s]i. sBya lulosiwng adisffoerleidnt dpiaaragmneotesriss tphaetr feontrimreleyd debsycritbhe ethFeP foDoStpSr.inFt,i gthue rveal4ueds eosbctraiinbeeds the lines
during the analysis allow a solid diagnosis performed by the FPDSS. Figure 4 describes the lines
mentionedaboveforthecalculationofthedifferentparameters.
mentioned above for the calculation of the different parameters.

Figure 4. Footprint parameters used by the FPDSS to evaluate the deformities in the footprint:
Figure 4. Footprint parameters used by the FPDSS to evaluate the deformities in the footprint:
(a) Staheli index; (b) Clarke’s angle; (c) Chippaux-Smirak index.
(a)Staheliindex;(b)Clarke’sangle;(c)Chippaux-Smirakindex.
2.3. Algorithms for Estimation of Measures in Footprint
The proposed algorithm is based on the application of image processing techniques. The
programming language used for the development of this algorithm was C. The design of a graphical
user interface (GUI) was performed with the use of GTK+ for a friendly user interaction with the
results and the algorithm. These methods were implemented using the OpenCV library [19] for the
manipulation of the footprint images.
In the first part, the camera acquires an 8-bit image in JPEG format with dimensions of 1920 ×
1080 pixels and RGB color space. Once the image is introduced in the software, several preprocessing
steps are made. Firstly, the RGB color image was converted into CIELAB (CIE 1976 L*a*b*) color
space. CIELAB provides a better interpretation of the image for the algorithm since the luminosity is
separated in a single channel and totally ignored in the color channels (a* and b*). The output
channels of the CIELAB model gave a better correlation with the visual interpretation of the footprint
images, something that is of great importance when the area of the footprint needs to be separated
from the skin of the foot. After that, a range of values in the a* and b* channel was tested for binarizing
the image based on an image thresholding step. The thresholding for a* channel was set to 122 and
for b* channel was set to 131, since that combination of thresholds correspond to any intensity of the
yellow and red color in the CIELAB color space. That means that any intensity above the thresholds
will be set as white pixels (1) and the rest will be black pixels (0), the range of intensity per pixel goes
Sensors2017,17,2700 6of17
2.3. AlgorithmsforEstimationofMeasuresinFootprint
The proposed algorithm is based on the application of image processing techniques.
The programming language used for the development of this algorithm was C. The design of a
graphicaluserinterface(GUI)wasperformedwiththeuseofGTK+forafriendlyuserinteractionwith
theresultsandthealgorithm. ThesemethodswereimplementedusingtheOpenCVlibrary[19]for
themanipulationofthefootprintimages.
In the first part, the camera acquires an 8-bit image in JPEG format with dimensions
of 1920×1080 pixels and RGB color space. Once the image is introduced in the software,
several preprocessing steps are made. Firstly, the RGB color image was converted into CIELAB
(CIE1976L*a*b*)colorspace. CIELABprovidesabetterinterpretationoftheimageforthealgorithm
sincetheluminosityisseparatedinasinglechannelandtotallyignoredinthecolorchannels(a*andb*).
TheoutputchannelsoftheCIELABmodelgaveabettercorrelationwiththevisualinterpretation
ofthefootprintimages,somethingthatisofgreatimportancewhentheareaofthefootprintneeds
tobeseparatedfromtheskinofthefoot. Afterthat,arangeofvaluesinthea*andb*channelwas
testedforbinarizingtheimagebasedonanimagethresholdingstep. Thethresholdingfora*channel
wassetto122andforb*channelwassetto131,sincethatcombinationofthresholdscorrespondto
anyintensityoftheyellowandredcolorintheCIELABcolorspace. Thatmeansthatanyintensity
abovethethresholdswillbesetaswhitepixels(1)andtherestwillbeblackpixels(0),therangeof
intensity per pixel goes from 0 to 255 in 8-bit images. Subsequently, the region of the footprint is
correctlysegmentedinabinaryimage. Then,thebinarizedimageofthepreviousstepisdilatedto
expandthesizeofthefootprintandtocoverallthesmallpartsthatremainedoutsidethefootduring
thethresholdingprocess. Finally,theimageissegmentedonlytogettheareathatweareinterested
in;thatistosay,thefootprintexcludingthetoes. Thiswasperformedusingafunctionthatdetects
thecontourofthebiggestobjectintheimageandtracesaboundarysothateverythingoutsidethe
contourissetasbackgroundorblackpixels.
After the preprocessing stage of the footprint is done, the algorithm looks for the one square
centimeterreferenceintheR(red)channeloftheimage,sinceitisthechanneloftheRGBcolormodel
whereitiseasiertoobtainthereference. Thisisaccomplishedwithafunctionthatiteratesoverthe
imageandlooksforcontinuousblackpixelsinthebottomandcountsthemtoknowtheareaofthe
square[20]. Thenumberofpixelsinonesquarecentimeterissavedforusingitwhenaconversion
frompixelstocentimetersisrequested.
Afterwards,thealgorithmfindsthekeypointsofthreeparametersintheprocessedimageused
for the analysis and detection of deformities in the footprint [1,21]. The parameters automatically
determinedbythisalgorithmaretheStaheliarchindex,Clarke’sangleandtheChippaux-Smirakindex.
Forthecalculationoftheparameters,thefootisdividedintothreemainareas:hindfoot,midfootand
forefootasshowninFigure5.Beforefindingtheparametersinthefootprint,afunctionisimplementedto
findifthereisarightfoot,leftfootorbothbyiteratingoverthebinarizedimagetofindthenumberof
contoursandtheircorrespondingposition.Byknowingwhichfootisontheimage,therightfunctionsare
appliedand,therefore,itmaximizestheprecisionofthecalculationanddiagnosis.Finally,thealgorithm
first obtains the measurement of the line in pixels, by using the location of the pixels at the starting
pointandtheendingpointwithEquation(1),andthentransformsittocentimetersusingthepreviously
calculatedreference:
(cid:113)
Z = (x −x )2+(y −y )2 (1)
2 1 2 1
Sensors 2017, 17, 2700 6 of 16
from 0 to 255 in 8-bit images. Subsequently, the region of the footprint is correctly segmented in a
binary image. Then, the binarized image of the previous step is dilated to expand the size of the
footprint and to cover all the small parts that remained outside the foot during the thresholding
process. Finally, the image is segmented only to get the area that we are interested in; that is to say,
the footprint excluding the toes. This was performed using a function that detects the contour of the
biggest object in the image and traces a boundary so that everything outside the contour is set as
background or black pixels.
After the preprocessing stage of the footprint is done, the algorithm looks for the one square
centimeter reference in the R (red) channel of the image, since it is the channel of the RGB color model
where it is easier to obtain the reference. This is accomplished with a function that iterates over the
image and looks for continuous black pixels in the bottom and counts them to know the area of the
square [20]. The number of pixels in one square centimeter is saved for using it when a conversion
from pixels to centimeters is requested.
Afterwards, the algorithm finds the key points of three parameters in the processed image used
for the analysis and detection of deformities in the footprint [1,21]. The parameters automatically
determined by this algorithm are the Staheli arch index, Clarke’s angle and the Chippaux-Smirak
index.
For the calculation of the parameters, the foot is divided into three main areas: hindfoot, midfoot
and forefoot as shown in Figure 5. Before finding the parameters in the footprint, a function is
implemented to find if there is a right foot, left foot or both by iterating over the binarized image to
find the number of contours and their corresponding position. By knowing which foot is on the
image, the right functions are applied and, therefore, it maximizes the precision of the calculation
and diagnosis. Finally, the algorithm first obtains the measurement of the line in pixels, by using the
location of the pixels at the starting point and the ending point with Equation (1), and then transforms
it to centimeters using the previously calculated reference:
Sensors2017,17,2700 7of17
Z = (cid:3493)((cid:1876) −(cid:1876) )(cid:2870)(cid:3397) ((cid:1877) −(cid:1877) )(cid:2870) (1)
(cid:2870) (cid:2869) (cid:2870) (cid:2869)

Figure 5. Divisions of the footprint in 3 zones.
Figure5.Divisionsofthefootprintin3zones.
The Staheli arch index was estimated by the division of the narrowest line in the midfoot (Q)
over Tthhee wStiadheeslti lainrceh inin tdheex hwinadsfoeostti m(Ra) t[e1d6]b. yThthee ndairvroiswioensto lfinthe einn athrreo mwiedsftoloint eisi ncatlhcuelmatieddf ofrootm(Q a)
ofuvnecrttiohne wthiadt eistetrlainteesi rnotwh ebhy irnodwfo fortom(R )ri[g1h6]t .toT hleeftn (afrorro lwefet sftoloint)e oirn lethfte tmo ridigfhoto (tfiosr craiglchutl afoteodt),f raonmd iat
ffuinndcst itohne tphiaxteilt ethraatte iss rfouwrthbeyr rforwomfr tohme srtiagrhttintog lpeoftin(fto (rinlenfetrf opooti)not rolre qft2t)o. Trihgehnt, (ifto drrraigwhst af ohootr),izaonndtaitl
filinned sththate gpoixees lftrhoamt itshfaut rpthoeinrtf rtoom thteh eensdta rotfi nthgep soeignmt(einnnteedr pfooointptroinrtq (2q)1.)T ahte tnh,ei tledfrta (wfosra lehfot rfiozootn) toarl
lriingehtt h(afotrg oriegshftr ofomott)h. aFtopro tihnet twoitdheesetn ldinoef itnh ethsee ghminednftoeodt,f othoetp fruinntc(tiqo1n) aist tthhee lseaftm(feo arsl eafbtofovoet,) bourtr itghhist
(tfimorer iigt hfitnfdoso tt)h.eF poirxtehl ethwaitd iess tthlein celoinsetshte ohnien fdrfoomot ,ththee sftaurntcintigo npoissitthioens aomf tehaes aalgboorvieth,bmu t(otuhitsert ipmoeinitt
fiorn dr2s) tahnedp tihxeel pthoaintti s(rt1h)e actl othsee setnodn eoff rtohme htohreizsotanrttainl glinpeo sthitrioonugohf tthhee ahlignodrfiothomt. F(oiguuterre p6o iilnlutsotrrart2e)sa tnhde
tphreocpeosisn dte(rs1cr)iabtedth aebeonvde.o fthehorizontallinethroughthehindfoot. Figure6illustratestheprocess
d eSesncsroirbs e2d017a,b 1o7,v 2e7.00 7 of 16
q q Q Q
1 1
q2 q2
r r r r R R
1 2 2 1

Figure 6. The process of the Staheli arch index calculation.
Figure6.TheprocessoftheStaheliarchindexcalculation.
The Clarke’s angle is calculated by taking the angle between a line T, that goes from the outer
poinTth oef Cthlaer fkoer’esfoaontg tloe tihsec aoluctuelra pteodinbt yoft athkein hginthdefoaont,g alendb eat wlineee nS,a thliante gTo,esth farotmgo tehsef oroumtert hpeoionut toefr
pothine tfoofretfhoeofto arnedfo tohtet ointnheer opuotienrt poof itnhteo mftidhfeohoitn [d22fo].o Tt,haen ldinae lTin ies Sca,ltchualtagteode sbyfr odmointhg eaonu itteerraptiooinn,t ionf
thtehefo froerfeofootota nanddt hheinidnfnoeort,p poiixnetl obfyt hpeixmeli dfrfoomot r[i2g2h]t. tToh leeflti n(feorT liesftc faolcout)l aotre ldefbt ytod roiignhgt a(fnori treirgahtti ofnoo,tin)
thteo ffoinredf othoet apnixdelh (int1d) ftohoatt, ipsi xtheel bcylopseixste ltof rtohme srtiagrhtitntgo pleofitn(tf oinr lbeoftthfo aorte)aosr. Tlehfet tloinrei gSh its (cfoalrcruilgahtetdfo boyt )
todfiranwdinthge ap liixneel f(rt1o)mt htahte iosuthteer cploosinets tinto ththe efosrtaerfotiontg (tp1o),i nthtaint wboasth foaurenads .inT htheel ipnreevSioisucsa sltceupl,a tteod thbey
drinanweirn gpoainlitn oef ftrhoem mthidefoooutt e(rs1p).o iAnftteinr dthreawfoinregf oaontd( te1s)t,imthaattinwga sthfeo umnedasiunrtehmeepnrte ovfi obuosths tleipn,esto, tthhee
inannegrlpe oeisnttimofatthioenm riedqfuoiortes(s 1a) .mAefatseurrdermawenint gofa nad theisrtdim laintien, gFt,h tehamt egaosuesr efmroemnt tohfeb oouthtelrin peos,intht eoaf ntghlee
eshtiinmdaftoioont (rfe1q) utoir tehsea inmneears puorienmt eonf tthoef matihdifrodotli (nse1,).F F,tihguatreg o7 esshforwoms tthhee lionuetse srppeociinfiteodf atbhoevhei.n Edqfuooatti(ofn1 )
(2) comes from the calculation of the angle of an isosceles triangle and it is used for Clark’s angle
estimation in this analysis:
S(cid:2870)− T(cid:2870)− F(cid:2870)
Clark’s Angle = cos(cid:2879)(cid:2869)( ) (2)
−2TF
t t T T
1 1
s s S S
1 1
F F
f f
1 1

Figure 7. The process of Clarke’s angle estimation.
The Chippaux-Smirak index requires the measurement of the widest line (P) of the forefoot
because we already have the narrowest line (Q) in the midfoot from the Staheli arch index calculation.
The algorithm iterates over the forefoot to find the widest line using outer (p1) and inner (p2) points
from the starting position of the algorithm (middle of the image), as shown in Figure 8. After having
calculated the measurements of both lines, the index is estimated by Equation (3) [18]:
Q
Chippaux−Smirakindex(%) = × 100 (3)
P
Sensors 2017, 17, 2700 7 of 16
q q Q Q
1 1
q2 q2
r r r r R R
1 2 2 1

Figure 6. The process of the Staheli arch index calculation.
The Clarke’s angle is calculated by taking the angle between a line T, that goes from the outer
point of the forefoot to the outer point of the hindfoot, and a line S, that goes from the outer point of
the forefoot and the inner point of the midfoot [22]. The line T is calculated by doing an iteration, in
the forefoot and hindfoot, pixel by pixel from right to left (for left foot) or left to right (for right foot)
to find the pixel (t1) that is the closest to the starting point in both areas. The line S is calculated by
Sednrsaorws2in01g7 ,a1 7l,in27e0 0from the outer point in the forefoot (t1), that was found in the previous step, to8 tohfe1 7
inner point of the midfoot (s1). After drawing and estimating the measurement of both lines, the
angle estimation requires a measurement of a third line, F, that goes from the outer point of the
totheinnerpointofthemidfoot(s1). Figure7showsthelinesspecifiedabove. Equation(2)comes
hindfoot (f1) to the inner point of the midfoot (s1). Figure 7 shows the lines specified above. Equation
fromthecalculationoftheangleofanisoscelestriangleanditisusedforClark’sangleestimationin
(2) comes from the calculation of the angle of an isosceles triangle and it is used for Clark’s angle
thisanalysis:
estimation in this analysis:
S2−T2−F2
CCllaarrkk(cid:48)’ss AAnnggllee==ccooss(cid:2879)−(cid:2869)(1S((cid:2870)−−T2(cid:2870)T−FF(cid:2870) )) (2)( 2)
−2TF
t t T T
1 1
s s S S
1 1
F F
f f
1 1

Figure 7. The process of Clarke’s angle estimation.
Figure7.TheprocessofClarke’sangleestimation.
The Chippaux-Smirak index requires the measurement of the widest line (P) of the forefoot
becaTuhsee wCeh iaplrpeaaudxy- Shmavier athkei nnadrerxowreeqsut liirnees (tQhe) inm tehaes muriedmfoeont tfroofmth theew Sitdaheestli lainrceh (iPn)deoxf cthalecufolarteifoono. t
bTechaeu aslegowreithalmre aitderyahteasv oevtehre tnhaer froorwefeosottl itnoe fi(nQd) tihnet hweidmeisdt floinoet furosimngt hoeutSetra h(pe1li) aarncdh iinnndeerx (cpa2l)c uploaitniotsn .
Tfhreomal gthoer isthtamrtiintegr patoessitoiovne rotfh tehefo arlegfooroitthtmo fi(mndidtdhlee wofi dtheest imlinaegeu)s,i ansg sohuotwern( ipn1 F)iagnudrei n8n. Aerft(epr2 h)apvoiinngt s
frcoamlcutlhaetesdta trhtien mgepaossuitrieomneonftsth oef ablogtohr iltihnmes, (tmheid idndleeox fisth eestiimmaagteed), bays sEhqouwatnioinn (F3i)g [u1r8e]:8 . Afterhaving
calculatedthemeasurementsofbothlines,theindexisestimatedbyEquation(3)[18]:
Q
Chippaux−Smirakindex(%) = × 100 (3)
P
Q
Chippaux−Smirakindex(%) = ×100 (3)
P
Sensors 2017, 17, 2700 8 of 16

p p2 p2 P P
1 p
1
q1 q1 Q Q
q2 q2

Figure 8. Process of the Chippaux-Smirak index calculation.
Figure8.ProcessoftheChippaux-Smirakindexcalculation.
Finally, the algorithm saves the acquired parameters information for further analysis and gives
the Fpionsaslilby,iltihtye atlog oarditdh mneswav elisntehse ianc qtuhier eidmpaagrea mweitther sainn faourmtoamtiaotnicf ocralfcuurltahteiorna noafl ytshise adnidstgainvcees tihne
pcoesnstibimiliettyertso. aInd dFingeuwre l9in, tehsei nretshueltism oafg eeacwhi tphaartn oaf uthtoem alagtoicrictahlmcu alartei osnhoowfnth veidsuisatlalyn.c e incentimeters.
InFigure9,theresultsofeachpartofthealgorithmareshownvisually.

Figure 9. Steps of the algorithm for the analysis of the footprint: (a) RGB image obtained from
podoscope; (b) Thresholding at 183 of red channel; (c) Segmentation of the footprint and removal of
toes; (d) Location of the reference (square centimeter) in the image; (e) Staheli arch index; (f) Clarke’s
angle; (g) Chippaux-Smirak index.
2.4. Graphic User Interface
The graphic user interface (GUI) was developed to provide a friendly tool for the user to
quantitative analyze and measure the footprint images by implementing the proposed algorithm.
The first step consists in using the camera for acquiring the footprint image and, once it is selected,
Sensors 2017, 17, 2700 8 of 16
p p2 p2 P P
1 p
1
q1 q1 Q Q
q2 q2

Figure 8. Process of the Chippaux-Smirak index calculation.
Finally, the algorithm saves the acquired parameters information for further analysis and gives
Stehnseo rsp2o0s1s7i,b1i7l,i2ty70 0to add new lines in the image with an automatic calculation of the distanc9e oifn1 7
centimeters. In Figure 9, the results of each part of the algorithm are shown visually.

FFigiguurree 99.. SStteeppss ooff tthhee aallggoorriitthhmm ffoorr tthhee aannaallyyssiiss ooff tthhee ffoooottpprriinntt:: ((aa)) RRGGBB iimmaaggee oobbtatainineedd frforomm
ppooddooscsocoppee;;( b(b))T Thrhersehsohlodlidnignga ta1t 8138o3f orfe drecdh cahnannenl;e(lc;) (Sc)e gSmegemnteanttioatnioonf tohfe tfhoeo ftoporitnptrianntd arnedm roevmaolvofalt ooefs ;
(dto)eLso; c(dat)i Lonocoaftitohne oref ftehree nrecfeer(seqnucea r(esqcuenartiem ceentetrim)ientethr)e iinm tahgee i;m(ea)gSe;t a(he)e lSitaarhcehli ianrdcehx i;n(df)eCx;l a(fr)k Ce’lsarakneg’sle ;
(ga)nCglhei;p (gp)a uCxh-iSpmpairuaxk-Sinmdireaxk. index.
22.4.4..G GrraapphhicicU Usseerr IInntteerrffaaccee
TThhee ggrraapphhiicc uusseerr iinntteerrffaaccee ((GGUUII)) wwaass ddeevveellooppeedd ttoo pprroovviiddee aa ffrriieennddlyly totoool lfoforr ththee uuseser rtoto
quantitative analyze and measure the footprint images by implementing the proposed algorithm.
quantitative analyze and measure the footprint images by implementing the proposed algorithm.
The first step consists in using the camera for acquiring the footprint image and, once it is selected,
Thefirststepconsistsinusingthecameraforacquiringthefootprintimageand,onceitisselected,

thealgorithmprocessestheimageandgivestheoutputbyautomaticallydrawingtheaforementioned
lines,withtheirrespectivemeasurementsincentimeters,andcalculatingthethreeparameters.
AsshowninFigure10,theGUIhassomeextrafunctionsthatgivetheuserthepossibilitytoadd
newlines,ormodifyandremovelinesthatarealreadydrawnintheimage. Thisisperformedbecause
theimageisplacedinthebottomofadrawingareawheretheusercantakemanuallymeasurements
usingamouse. Thesoftwarerecordsthecoordinatesofthestartingpointandtheendingpointofthe
newlinesotheconversionfrompixelstocentimeterscanbeperformed. Also,itrecognizeswhena
clickisgiveninaspecificlinesoitcanbeeithermodifiedorerased. Therearefourtabsonthetopof
theimage;eachoneincludesdifferentlinesdrawnbythesoftware. Forexample,thefirstoneincludes
all the lines that the algorithm used for the detection of the three parameters. The rest of the tabs
includejustthelinesusedfortheestimationoftheindexindicatedinitsowntitle.
Sensors 2017, 17, 2700 9 of 16
the algorithm processes the image and gives the output by automatically drawing the
aforementioned lines, with their respective measurements in centimeters, and calculating the three
parameters.
As shown in Figure 10, the GUI has some extra functions that give the user the possibility to add
new lines, or modify and remove lines that are already drawn in the image. This is performed because
the image is placed in the bottom of a drawing area where the user can take manually measurements
using a mouse. The software records the coordinates of the starting point and the ending point of the
new line so the conversion from pixels to centimeters can be performed. Also, it recognizes when a
click is given in a specific line so it can be either modified or erased. There are four tabs on the top of
the image; each one includes different lines drawn by the software. For example, the first one includes
aSleln stohrse2 0li1n7e,1s7 ,th27a0t0 the algorithm used for the detection of the three parameters. The rest of the10 toafb1s7
include just the lines used for the estimation of the index indicated in its own title.

Figure 10. The graphical user interface of the footprint analysis sensor.
Figure10.Thegraphicaluserinterfaceofthefootprintanalysissensor.
The parameters that result from the analysis of the footprint can be plotted and saved over time
to graTphheicpaalrlya mobesteerrsveth tahter iemsuplrtofvroemmetnhte oafn tahley spiastoiefntth, easfo doetppircintetdc ainn Fbieguprloe t1te1d, aanndd tshaev eefdfeoctvievrentiemsse
otof tghrea pphreicsaclrliybeodb sterrevaetmtheenti.m Tphreorvee amree nthtroefet hpeloptsa,t ioennet ,faosr deaepchic itneddeixn, Faingdu rtehe1 1in,adnedx cthaen ebfefe ocbtisveernveesds
foofrt ehaecphr feosocrt iibneddetpreenadtmenetnlyt.. TFhinearellya,r ethteh rueseepr lhoatss, tohnee pfoosrseibaiclhityin tdoe exx,paonrdt aths ePiDndF eoxr cparninbte thoeb siemrvagede
wfoirthe athche fdoroatwinnd lienpeesn. d ently. Finally,theuserhasthepossibilitytoexportasPDForprinttheimage
withthedrawnlines.