Modeling Markov Switching ARMA-GARCH Neural Networks Models and an Application to ...

22 Pages·2015·1.66 MB·English

Hindawi Publishing Corporation
e Scientific World Journal
Volume 2014, Article ID 497941, 21 pages
http://dx.doi.org/10.1155/2014/497941
Research Article
Modeling Markov Switching ARMA-GARCH Neural Networks
Models and an Application to Forecasting Stock Returns
MelikeBildirici1andÖzgürErsin2
1YıldızTechnicalUniversity,DepartmentofEconomics,BarbarosBulvari,Besiktas,34349Istanbul,Turkey
2BeykentUniversity,DepartmentofEconomics,Ayazag˘a,S¸i¸sli,34396Istanbul,Turkey
CorrespondenceshouldbeaddressedtoMelikeBildirici;[email protected]
Received20August2013;Accepted4November2013;Published6April2014
AcademicEditors:T.Chen,Q.Cheng,andJ.Yang
Copyright©2014M.BildiriciandO¨.Ersin.ThisisanopenaccessarticledistributedundertheCreativeCommonsAttribution
License,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperly
cited.
Thestudyhastwoaims.ThefirstaimistoproposeafamilyofnonlinearGARCHmodelsthatincorporatefractionalintegration
andasymmetricpowerpropertiestoMS-GARCHprocesses.ThesecondpurposeofthestudyistoaugmenttheMS-GARCHtype
modelswithartificialneuralnetworkstobenefitfromtheuniversalapproximationpropertiestoachieveimprovedforecasting
accuracy.Therefore,theproposedMarkov-switchingMS-ARMA-FIGARCH,APGARCH,andFIAPGARCHprocessesarefurther
augmentedwithMLP,RecurrentNN,andHybridNNtypeneuralnetworks.TheMS-ARMA-GARCHfamilyandMS-ARMA-
GARCH-NNfamilyareutilizedformodelingthedailystockreturnsinanemergingmarket,theIstanbulStockIndex(ISE100).
ForecastaccuracyisevaluatedintermsofMAE,MSE,andRMSEerrorcriteriaandDiebold-Marianoequalforecastaccuracytests.
TheresultssuggestthatthefractionallyintegratedandasymmetricpowercounterpartsofGray’sMS-GARCHmodelprovided
promisingresults,whilethebestresultsareobtainedfortheirneuralnetworkbasedcounterparts.Further,amongthemodels
analyzed,themodelsbasedontheHybrid-MLPandRecurrent-NN,theMS-ARMA-FIAPGARCH-HybridMLP,andMS-ARMA-
FIAPGARCH-RNNprovidedthebestforecastperformancesoverthebaselinesingleregimeGARCHmodelsandfurther,overthe
Gray’sMS-GARCHmodel.Therefore,themodelsarepromisingforvariouseconomicapplications.
1.Introduction on volatility, the financial returns are under the influence
of sudden or abrupt changes in the economy. Hence, the
Inthelightofthesignificantimprovementsintheeconomet- volatilityofeconomicdatahasbeenexploredineconometric
ric techniques and in the computer technologies, modeling literature as a result of the need of modelling uncertainty
the financial time series have been subject to accelerated and risk in the financial returns. The relationship between
empiricalinvestigationintheliterature.Accordingly,follow- the financial returns and various important factors such as
ing the developments in the nonlinear techniques, analyses the trade volume, market price of financial assets, and the
focusingonthevolatilityinfinancialreturnsandeconomic relationship between volatility, trade volume, and financial
variables are observed to provide significant contributions. returnshavebeenvigorouslyinvestigated[1–4].
It could be stated that important steps have been taken TheARCHmodelintroducedbyEngle[5]andtheGener-
in terms of nonlinearmeasurement techniques focusing on alizedARCH(GARCH)modelintroducedbyBollerslev[6]
the instability or stability occurring vis-a-vis encountered are generally accepted for measuring volatility in financial
volatility. Further, the determination of stability or insta- models. GARCH models have been used intensively in
bility in terms of volatility in the financial markets gains academicstudies.AtremendousamountofGARCHmodels
importanceespeciallyforanalyzingtheriskencountered.In existandvariousstudiesprovideextendedevaluationofthe
addition to impact of the magnitude and the size of shocks development.
2 TheScientificWorldJournal
Among many, Engle and Bollerslev [7] developed the cited studies discussing high persistence in volatilitydue to
Integrated-GARCH(I-GARCH)processtoincorporateinte- structural changes. Lamoureux and Lastrapes [24] showed
gration properties, AGARCH model, introduced by Engle thattheencounteredhighpersistenceinvolatilityprocesses
[8], allows modeling asymmetric effects of negative and resultedfromthevolumeeffectsthathadnotbeentakeninto
positive innovations. In terms of modeling asymmetries, account.QiaoandWong[25]followedabivariateapproach
GARCH models have been further developed by including andconfirmedthattheLamoureuxandLastrapes[24]effect
asymmetric impacts of the positive and negative shocks
existsduetothevolumeandturnovereffectsonconditional
to capture the asymmetric effects of shocks on volatility
volatility and after the introduction of volume/turnover as
and return series which depend on the type of shocks, i.e.
exogenous variables, it is possible to obtain a significant
either negative or positive. Following the generalization of
decline in the the persistence. Mikosch and Staˇricaˇ [26]
EGARCH model of Nelson [9] that allows modelling the
showedthatstructuralchangeshadanimportantimpactthat
asymmetriesintherelationshipbetweenreturnandvolatility,
leads to accepting an integrated GARCH process. Bauwens
the Glosten et al. [10] noted the importance of asymmetry
etal.[27,28]discussedthatthepersistenceintheestimated
causedbygoodandbadnewsinvolatileseriesandproposeda
single regime GARCH processes could be considered as
modelthatincorporatesthepastnegativeandpositiveinno-
resulting from the misspecification which could be con-
vations with an identity function that leads the conditional
trolled by introducing an MS-GARCH specification where
variancetofollowdifferentprocessesduetoasymmetry.The
theregimeswitchesaregovernedbyahiddenMarkovchain.
finding is a result of the empirical analyses which pointed
Kra¨mer[29]evaluatedtheautocorrelationinthesquared
at the fact that the negative shocks had a larger impact on
errortermsandprovidedanimportantcontribution.Accord-
volatility. Consequently, the bad news have a larger impact ingly, the observed empirical autocorrelations of the 𝜀2 are
compared to the conditional volatility dynamics followed 𝑡
muchlargerthanthetheoreticalautocorrelationsimpliedby
afterthegoodnews.Duetothiseffect,asymmetricGARCH
theestimatedparametersthroughevaluatinganMS-GARCH
modelshaverapidlyexpanded.TheGJR-GARCHmodelwas
modelwheretheautocorrelationproblemcouldbeshownto
developedindependentlybyZakoian[11,12]andGlostenet
accelerate as the transition probabilities approached 1. (For
al.[10].Itshouldbenotedthat,intermsofasymmetry,the
a proof see e.g., Francq and Zako¨ıan [30]; Kra¨mer [29])
ThresholdGARCH(T-GARCH)ofZakoian[12],VGARCH,
(Inparticular,theempiricalautocorrelationsofthe𝜀2 often
and nonlinear asymmetric GARCH models (NAGARCH) 𝑡
seem to indicate longmemory,which is notpossible in the
of Engle and Ng [13] are closely related versions to model
GARCH-model; in fact, in all standard GARCH-models,
asymmetry in financial asset returns. The SQR-GARCH
theoretical autocorrelations must eventually decrease expo-
modelofHestonandNandi[14]andtheAug-GARCHmodel
nentially,solongmemoryisruledout).AlexanderandLazaar
developedbyDuan[15]nestseveralversionsofthemodels
[31]showedthatleverageeffectsareduetoasymmetryinthe
taking asymmetry discussed above. Further, models such
volatilityresponsestothepriceshocksandtheleverageeffect
as the Generalized Quadratic GARCH (GQARCH) model
acceleratesoncethemarketsareinthemorevolatileregime
of Sentana [16] utilize multiplicative error terms to capture
Kra¨merandTameze[32]showedthatasinglestateGARCH
volatilitymoreeffectively.TheFIGARCHmodelofBaillieet
modelhadonlyonemeanreversionwhilebyallowingregime
al.[17]benefitsfromanARFIMAtypefractionalintegration
switching in the GARCH processes, mean reverting effect
representationtobettercapturethelong-rundynamicsinthe
diminishes. In a perspective of volatility, if these shifts are
conditionalvariance(Seefordetailedinformation,Bollerslev
persistent,thentherearetwosourcesofvolatilitypersistence,
[18]).TheAPARCH/APGARCHmodelofDingetal.[19]is
duetoshocksandduetoregime-switchingintheparameters
an asymmetric model that incorporates asymmetric power
ofthevarianceprocess.ByutilizingaMarkovtransformation
terms which are allowed to be estimated directly from the
model, it could be shown that the relationships among the
data.TheAPGARCHmodelalsonestsseveralmodelssuch
regimesbetweentheperiodsof𝑡−1and𝑡couldbeexplained
as the TGARCH, TSGARCH, GJR, and logGARCH. The
andthemostimportantadvantageoftheMSGARCHmodel
FIAPGARCHmodelofTse[20]combinestheFIGARCHand
exposesitselfasthereisnoneedfortheresearcherstoobserve
the APGARCH. Hyperbolic GARCH (HYGARCH) model
the regime changes. The model allows different regimes to
of Davidson [21] nests the ARCH, GARCH, IGARCH, and
revealbyitself[33].
FIGARCHmodels(foranextendedreviewGARCHmodels,
seeBollerslev[18]). The regime switching in light of the Markov switching
EventhoughtheARCH/GARCHmodelscanbeapplied model has interesting properties to be examined such as
quickly for many time series, the shortcomings in these thestationaritybyallowingtheswitchingcourseofvolatility
models were discussed by certain studies. Perez-Quiros inherent in the asset prices. The hidden Markov model
and Timmermann [22] focused on the conditional distri- (HMM)developedbyTaylor[34]isaswitchingmodelthat
butions of financial returns and showed that recessionary benefits from including an unobserved variable to capture
and expansionary periods possess different characteristics, volatilitytobemodeledwithtransitionsbetweenthehidden
while the parameters of a GARCH model are assumed to statesthatpossessdifferentprobabilitydistributionsattached
bestableforthewholeperiod.Certainstudiesdiscussedthe to each state. Hidden Markov model has been applied
highvolatilitypersistenceinheritedinthebaselineGARCH successfully by Alexander and Dimitriu [35], Cheung and
and proposed early signs of regime switches. Diebold [23] Erlandsson [36], Francis and Owyang [37], and by Clarida
and Lamoureux and Lastrapes [24] are two of the highly et al. [38] to capture the switching type of predictions in
TheScientificWorldJournal 3
stock returns, interest rates, and exchange rates. Regime versions: the MS-ARMA-GARCH-HybridNN, MS-ARMA-
switching model has been used extensively for prediction APGARCH-HybridNN, MS-ARMA-FIGARCH-HybridNN,
of returns belonging to different stock market returns in andMS-ARMA-FIAPGARCH-HybridNN.
differenteconomiesandbyfollowingthefactthatthestock
market indices are very sensitive to stock volatility, which
acceleratesespeciallyduringperiodswithmarketturbulences 2.TheMS-GARCHModels
(seefordetailedinformation,AlexanderandKaeck,[39]).
Over long periods, there are many reasons why financial
The conventional statistical techniques for forecasting
seriesexhibitimportantbreaksinbehavior;examplesinclude
reached their limit in applications with nonlinearities, fur-
depression, recession, bankruptcies, natural disasters, and
thermore,recentresultssuggestthatnonlinearmodelstend
market panics, as well as changes in government policies,
to perform better in models for stock returns forecasting
investor expectations, or the political instability resulting
[40]. For this reason, many researchers have used artificial
fromregimechange.
neuralnetworksmethodologiesforfinancialanalysisonthe
Diebold[23]providedathroughoutanalysisonvolatility
stockmarket.LaiandWong[41]contributedtothenonlinear models.Oneoftheimportantfindingsisthefactthatvolatil-
timeseriesmodelingmethodologybymakinguseofsingle- itymodelsthatfailtoadequatelyincorporatenonlinearityare
layer neural network. Further, modeling of NN models for subjecttoanupwardbiasintheparameterestimateswhich
estimation and prediction for time series have important results in strong forms of persistence that occurs especially
contributions. Weigend et al. [42], Weigend and Gershen- inhighvolatilityperiodsinfinancialtimeseries.Asaresult
feld [43], White [44], Hutchinson et al. [45], and Refenes ofthebiasintheparameterestimates,oneimportantresult
et al. [46] contributed to financial analyses, stock market ofthisfactisontheout-of-sampleforecastsofsingleregime
returns estimation, pattern recognition, and optimization. typeGARCHmodels.Accordingly,Schwert[66]proposeda
NNmodelingmethodologyisappliedsuccessfullybyWang model that incorporates regime switching that is governed
et al. [47] and Wang [48] to forecast the value of stock by a two state Markov process, hence the model retains
indices. Similarly, Abhyankar et al. [49], Castiglione [50], differentcharacteristicsintheregimesthataredefinedashigh
Freisleben[51],KimandChun[52],LiuandYao[53],Phua volatilityandlowvolatilityregimes.
Hamilton [67] proposed the early applications of HMC
etal.[54],Refenesetal.[55],Resta[56],R.SitteandJ.Sitte
modelswithinaMarkovswitchingframework.Accordingly,
[57],Tinˇoetal.[58],YaoandPoh[59],andYaoandTan[60]
MS models were estimated by maximum likelihood (ML)
are important investigations focusing on the relationships
wheretheregimeprobabilitiesareobtainedbytheproposed
between stock prices and market volumes and volatility.
Hamilton-filter[68–71].MLestimationofthemodelisbased
For similar applications, see [1–4]. Bildirici and Ersin [61]
on a version of the Expectation Maximization (EM) algo-
modeledNN-GARCHfamilymodelstoforecastdailystock
rithmasdiscussedinHamilton[72],Krolzig[73–76].Inthe
returnsforshortandlongrunhorizonsandtheyshowedthat
MSmodels,regimechangesareunobservedandareadiscrete
GARCH models augmented with artificial neural networks
state of a Markov chain which governs the endogenous
(ANN) architectures and algorithms provided significant
switches between different AR processes throughout time.
forecasting performances. Ou and Wang [62] extended the
By inferring the probabilities of the unobserved regimes
NN-GARCH models to Support Vector Machines. Azadeh
whichareconditionalonaninformationset,itispossibleto
etal.[63]evaluatedNN-GARCHmodelsandproposedthe reconstructtheregimeswitches[77].
integrated ANN models. Bahrammirzaee [64] provided an Furthermore, certain studies aimed at the development
analysisbasedonfinancialmarketstoevaluatetheartificial of modeling techniques which incorporate both the proba-
neuralnetworks,expertsystems,andhybridintelligencesys- bilisticpropertiesandtheestimationofaMarkovswitching
tems.Further,KanasandYannopoulos[65]andKanas[40] ARCHandGARCHmodels.Aconditionforthestationarity
usedMarkovswitchingandNeuralNetworkstechniquesfor of a natural path-dependent Markov switching GARCH
forecasting stock returns; however, their approaches depart model as in Francq et al. [78] and a throughout analysis
fromtheapproachfollowedwithinthisstudy. oftheprobabilisticstructureofthatmodel,withconditions
Inthisstudy,theneuralnetworksandMarkovswitching for the existence of moments of any order, are developed
structuresareaimedtobeintegratedtoaugmenttheARMA- and investigated in Francq and Zako¨ıan [30]. Wong and
GARCH models by incorporating regime switching and Li [79], Alexander and Lazaar [80], and Haas et al. [81–
different neural networks structures. The approach aims at 83] derived stationarity analysis for some mixing models
formulations and estimations of MS-ARMA-GARCH-MLP, of conditional heteroskedasticity [27, 28]. For the Markov
MS-ARMA-APGARCH-MLP, MS-ARMA-FIGARCH-MLP, switchingGARCHmodelsthatavoidthedependencyofthe
MS-ARMA-FIAPGARCH-MLP, MS-ARMA-GARCH-RBF conditional variance on the chain’s history, the stationarity
MS-ARMA-APGARCH-RBF, MS-ARMA-FIGARCH-RBF, conditionsareknownforsomespecialcasesintheliterature
and MS-ARMA-FIAPGARCH-RBF; the recurrent neural [84].Klaassen[85]developedtheconditionsforstationarity
network augmentationsof the models are, namely, the MS- of the model as the special cases of the two regimes.
ARMA-GARCH-RNN MS-ARMA-APGARCH-RNN, MS- A necessary and sufficient stationarity condition has been
ARMA-FIGARCH-RNN, and MS-ARMA-FIAPGARCH- developedbyHaasetal.[81–83]fortheirMarkovswitching
RNN. And lastly, the paper aims at providing Hybrid NN GARCHmodel.Furthermore,Cai[86]showedtheproperties
4 TheScientificWorldJournal
ofBayesianestimationofaMarkovswitchingARCHmodel conditionalvariancesareinthesameregime(fordetails,the
where only the constant in the ARCH equation is allowed readersarereferredtoBauwensetal.[27,28],Klaassen[85],
tohaveregimeswitches.Theapproachhasbeeninvestigated Haas et al. [81–83], Francq and Zako¨ıan [30], Kra¨mer [29],
by Kaufman and Fru¨hwirth-Schnatter [87] and Kaufmann andAlexanderandKaeck[39]).
and Scheicher [88]. Das and Yoo [89] proposed an MCMC Another area of analysis pioneered by Haas [97] and
algorithmforthesamemodel(switchesbeingallowedinthe Chang et al. [98] allow different distributions in order to
constantterm)withasinglestateGARCHtermtoshowthat gainforecastaccuracy.Animportantfindingofthesestudies
gainscouldbeachievedtoovercomepath-dependence.MS- showedthatbyallowingtheregimedensitiestofollowskew-
GARCHmodelsarestudiedbyFrancqandZako¨ıan[30]to normaldistributionwithGaussiantailcharacteristics,several
achievetheirnon-Bayesianestimationpropertiesinlightof return series could be modeled more efficiently in terms
thegeneralizedmethodofmoments.Bauwensetal.[27,28] of forecast accuracy. Liu [99] developed and discussed the
proposed a Bayesian Markov chain Monte Carlo (MCMC) conditions for stationarity in Markov switching GARCH
algorithmthatisdifferentiatedbyincludingthestatevariables structure in Haas et al. [81–83] and proved the existence
in the parameter space to control the path-dependence by of the moments. In addition, Abramson and Cohen [100]
obtainingtheparameterspacewithGibbssampling[90]. discussed and further evaluated the stationarity conditions
ThehighandlowvolatilityprobabilitiesofMS-GARCH in a Markov switching GARCH process and extended the
modelsallowdifferentiatinghighandlowvolatilityperiods. analysis to a general case with m-state Markov chains and
By observing the periods in which volatility is high, it is GARCH(𝑝,𝑞) processes. An evaluation and extension of
possibletoinvestigatetheeconomicandpoliticalreasonsthat the stationarity conditions for a class of nonlinear GARCH
caused increased volatility. If a brief overview is to be pre- models are investigated in Abramson and Cohen [100].
sented,thereareseveralmodelsbasedontheideaofregime Francq and Zako¨ıan [30] derived the conditions for weak
changeswhichshouldbementioned.Schwert[66]explores stationarity and existence of moments of any order MS-
amodelinwhichswitchesbetweenthesestatesthatreturns GARCH model. Bauwens et al. [27, 28] showed that by
canhaveahighorlowvariancearedeterminedbyatwo-state enlarging the parameter space to include space variables,
Markovprocess.LamoureuxandLastrapes[24]suggestthe though maximum likelihood estimation is not feasible, the
useofMarkovswitchingmodelsforawayofidentifyingthe Bayesianestimationoftheextendedprocessisfeasiblefora
timingoftheshiftsintheunconditionalvariance.Hamilton modelwheretheregimechangesaregovernedwithahidden
and Susmel [91] and Cai [86] proposed Markov switching Markov chain. Further, Bauwens et al. [27, 28] accepted
ARCH model to capture the effects of sudden shifts in the mildregularityconditionsunderwhichtheMarkovchainis
conditional variance. Further, Hamilton and Susmel [91] geometricallyergodicandhasfinitemomentsandisstrictly
extended the analysis to a model that allows three regimes, stationary.
which were differentiated between low, moderate and high
volatilityregimes,wherethehigh-volatilityregimecaptured 2.1.MS-ARMA-GARCHModels. Toavoidpath-dependence
theeconomicrecessions.Itisacceptedthattheproposalsof problem,Gray[92]suggestsintegratingouttheunobserved
Cai[86]andHamiltonandSusmel[91]helpedtheresearchers regime path in the GARCH term by using the conditional
tocontrolfortheproblemofpathdependence,whichmakes expectationofthepastvariance.Gray’sMS-GARCHmodel
thecomputationofthelikelihoodfunctionimpossible(The isrepresentedasfollows:
conditionalvarianceattimetdependsontheentiresequence
of regimes up to time t due to the recursive nature of the 𝜎2 =𝑤 +∑𝑞 𝛼 𝜀2 +∑𝑝 𝛽 𝐸( 𝜀𝑡2−𝑗 )
GARCHprocess.InMarkovswitchingmodel,theregimesare 𝑡,(𝑠𝑡) (𝑠𝑡) 𝑖=1 𝑖,(𝑠𝑡) 𝑡−𝑖 𝑗=1 𝑗,(𝑠𝑡) 𝐼𝑡−𝑗−1
unobservable,oneneedstointegrateoverallpossibleregime
paths.Thenumberofpossiblepathsgrowsexponentiallywith 𝑞
t, which renders ML estimation intractable.) (see for detail, =𝑤 +∑𝛼 𝜀2 (1)
(𝑠) 𝑖,(𝑠) 𝑡−𝑖
𝑡 𝑡
Bauwens,etal.[27,28]). 𝑖=1
Gray[92]studyisoneoftheimportantstudieswherea
𝑝 𝑚 𝑠
MarkovswitchingGARCHmodelisproposedtoovercome +∑𝛽 ∑𝑃(𝑠 = 𝑡−𝑗 )𝜎2 ,𝑠 ,
the path dependence problem. According to Gray’s model, 𝑗,(𝑠𝑡) 𝑡−𝑗 𝐼 𝑡−𝑗 𝑡−𝑗
𝑗=1 𝑠𝑡−𝑗=1 𝑡−𝑗−1
once the conditional volatility processes are differentiated
betweenregimes,anaggregationoftheconditionalvariances where 𝑤𝑠𝑡 > 0, 𝛼𝑖,𝑠𝑡 ≥ 0, 𝛽𝑗,𝑠𝑡 ≥ 0, and 𝑖 = 1,...,𝑞,
fortheregimescouldbeusedtoconstructasinglevariance 𝑗 = 1,...,𝑝, 𝑠𝑡 = 1,...,𝑚. The probabilistic structure of
coefficienttoevaluatethepathdependence.Amodification the switching regime indicator 𝑠𝑡 is defined as a first-order
is also conducted by Klaassen [85]. Yang [93], Yao and Markovprocesswithconstanttransitionprobabilities𝜋1and
Attali [94], Yao [95], and Francq and Zako¨ıan [96] derived 𝜋2, respectively (Pr{𝑠𝑡 = 1 | 𝑠𝑡−1 = 1} = 𝜋1, Pr{𝑠𝑡 = 2 |
conditions for the asymptotic stationarity of some AR and 𝑠𝑡−1 = 1} = 1 − 𝜋1, Pr{𝑠𝑡 = 2 | 𝑠𝑡−1 = 2} = 𝜋2, and
ARMA models with Markov switching regimes. Haas et al. Pr{𝑠𝑡 =1|𝑠𝑡−1 =2}=1−𝜋2).
[81–83]investigatedaMS-GARCHmodelbywhichafinite AlthoughDueker[101]acceptsacollapsingprocedureof
state-space Markov chain is assumed to govern the ARCH Kim’s [102] algorithm to overcome path-dependence prob-
parameters, whereas the autoregressive process followed by lem,Dueker[101]adoptsthesameframeworkofGray[92].
theconditionalvarianceissubjecttotheassumptionthatpast Accordingly,themodifiedGARCHversionofDueker[101]is
TheScientificWorldJournal 5
𝜀 =𝐸[𝜀 |𝑠 ,𝑌 ],
acceptedwhichgovernsthedispersioninsteadoftraditional 𝑡−𝑖−1,(𝑠 ) 𝑡−𝑖−1,(𝑠 ) 𝑡−𝑖 𝑡−𝑖−1
𝑡−𝑖 𝑡−𝑖−1
GARCH(1,1)specification. (3)
Yang [103], Yao and Attali [94], Yao [95], and Francq 𝜎𝑡−𝑖−1,(𝑠 ) =𝐸[𝜀𝑡−𝑖−1,(𝑠 ) |𝑠𝑡−𝑖,𝑌𝑡−𝑖−1].
𝑡−𝑖 𝑡−𝑖−1
and Zako¨ıan [96] derived conditions for the asymptotic
stationarity of models with Markov switching regimes (see Thus, the parameters have nonnegativity constraints
for detailed information Bauwens and Rombouts [104]. 𝜙,𝜃,𝜑,𝑤,𝛼,𝛽>0andtheregimesaredeterminedby𝑠𝑡,
The major differences between Markov switching GARCH
𝑇
ims,otdheelscoarneditthioensaplevcairfiicaantcioen𝜎𝑡2of=theVavra(r𝜀i𝑡a/n𝑆c𝑡e).pTroocceossn;sitdheart 𝐿=∏𝑡=1𝑓(𝑦𝑡 |𝑠𝑡 =𝑖,𝑌𝑡−1)Pr[𝑠𝑡 =𝑖|𝑌𝑡−1], (4)
theconditionalvarianceasintheBollerslev’s[105]GARCH
modelandtoconsidertheregimedependentequationforthe and the probability Pr[𝑠𝑡 = 𝑖 | 𝑌𝑡−1] is calculated through
conditionalvarianceinFro¨mmel[106]areacceptedthatThe iteration:
coefficients𝑤𝑠𝑡,𝛼𝑠𝑡,𝛽𝑠𝑡 correspondtorespectivecoefficients 𝜋𝑗𝑡 =Pr[𝑠𝑡 =𝑗|𝑌𝑡−1]
intheone-regimeGARCHmodel,butmaydifferdepending
onthepresentstate. 1 1
Klaassen [85] (Klassen [85] model is defined as 𝜎𝑡2,(𝑠) = =∑Pr[𝑠𝑡 =𝑗|𝑠𝑡−1 =𝑖]Pr[𝑠𝑡 =𝑗|𝑌𝑡−1]∑𝜂𝑗𝑖𝜋𝑖∗𝑡−1.
𝑤 +∑𝑞 𝛼 𝜀2 +∑𝑝 𝛽 ∑𝑚 𝑃(𝑆 =𝑠 |𝐼 ,𝑆𝑡 = 𝑖=0 𝑖=0
(𝑠𝑡) 𝑖=1 𝑖,(𝑠𝑡) 𝑡−𝑖 𝑗=1 𝑗,(𝑠𝑡) ̃𝑠=1 𝑡−𝑗 𝑡−𝑗 𝑡−1 𝑡 (5)
𝑠𝑡)𝜎𝑡2−𝑗,𝑠𝑡−𝑗) suggested to use the conditional expectation of
the lagged conditionalvariance with a broader information Accordingly,thetwomodels,theHennekeetal.[108]andthe
set than the model derived in Gray [92]. Accordingly, Francq et al. [78] approaches, could be easily differentiated
Klaassen [85] suggested modifying Gray’s [92] model by throughthedefinitionsof𝜀𝑡2−1and𝜎𝑡−1.Further,asymmetric
replacing𝑝(𝑠𝑡−𝑗 =𝑠𝑡−𝑗 |𝐼𝑡−𝑗−1) by𝑝(𝑠𝑡−𝑗 =𝑠𝑡−𝑗 |𝐼𝑡−1,𝑆𝑡 =𝑠𝑡) powertermsandfractionalintegrationwillbeintroducedto
whileevaluating 𝜎𝑡2,𝑠𝑡. thederivedmodelinthefollowingsections.
Another version of MS-GARCH model is developed by
Haas et al. [81–83]. According to this model, Markov chain 2.2.MS-ARMA-APGARCHModel. Liu[99]providedagen-
controlstheARCHparametersateachregime(𝑤𝑠,𝛼𝑖,𝑠)and eralizationoftheMarkovswitchingGARCHmodelofHaas
the autoregressive behavior in each regime is subject to the et al. [81–83] and derived the conditions for stationarity
assumptionthatthepastconditionalvariancesareinthesame and for the existence of moments. Liu [99] proposes a
regimeasthatofthecurrentconditionalvariance[100]. model which allowed for a nonlinear relation between past
In this study, models will be derived following the MS- shocksandfuturevolatilityaswellasfortheleverageeffects.
ARMA-GARCH specification in the spirit of Blazsek and The leverage effect is an outcome of the observation that
Downarowicz [107] where the properties of MS-ARMA- the reaction of stock market volatility differed significantly
GARCH processes were derived following Gray [92] and to the positive and the negative innovations. Haas et al.
Klaassen [85] framework. Henneke et al. [108] developed [109,110]complementsLiu’s[99]workintwoways.Firstly,
an approach to investigate the model derived in Francq the representation of the model developed by Haas [109]
et al. [78] for which the Bayesian framework was derived. allows computational ease for obtaining the unconditional
The stationarity of the model was evaluated by Francq and moments.Secondly,thedynamicautocorrelationstructureof
Zako¨ıan [96] and an algorithm to compute the Bayesian thepower-transformedabsolutereturns(residuals)wastaken
estimator of the regimes and parameters was developed. It asameasureofvolatility.
shouldbenotedthattheMS-ARMA-GARCHmodelsinthis Haas [109] model assumes that time series {𝜀𝑡,𝑡 ∈ Z}
paper were developed by following the models developed followsakregimeMS-APGARCHprocess,
in the spirit of Gray [92] and Klaassen [85] similar to the
𝜀 =𝜂𝜎 𝑡∈Z,
frameworkofBlazsekandDownarowicz[107]. 𝑡 𝑡 Δ𝑡,𝑡 (6)
TheMS-ARMA-GARCHmodelwithregimeswitchingin with {𝜂𝑡,𝑡 ∈ Z} being i.i.d. sequence and {Δ𝑡,𝑡 ∈ Z} is a
the conditional mean and variance are defined as a regime Markovchainwithfinitestatespace𝑆 = {1,...,𝑘}and𝑃is
switchingmodelwheretheregimeswitchesaregovernedby
the irreducible and aperiodic transition matrix with typical
anunobservedMarkovchainintheconditionalmeanandin element𝑝𝑖𝑗 =𝑝(Δ𝑡 =𝑗|Δ𝑡−1 =𝑖)sothat
theconditionalvarianceprocessesas
𝑃=[𝑝𝑖𝑗]=[𝑝(Δ𝑡 =𝑗|Δ𝑡−1 =𝑖)], 𝑖,𝑗=1,...,𝑘. (7)
𝑟 𝑚
𝑦 =𝑐 +∑𝜃 𝑦 +𝜀 +∑𝜑 𝜀 ,
𝑡 (𝑠) 𝑖,(𝑠) 𝑡−𝑖 𝑡,(𝑠) 𝑗,(𝑠) 𝑡−𝑗,(𝑠) The stationary distribution of Markov-chain is shown as
𝑡 𝑡 𝑡 𝑡 𝑡
𝑖=1 𝑗=1 𝜋∞ =(𝜋1,∞,...,𝜋𝑘,∞)󸀠.
(2)
𝑝 𝑞 According to the Liu [99] notation of MS-APGARCH
𝜎2 =𝑤 +∑𝛼 𝜀2 +∑𝛽 𝜎 , model, the conditional variance 𝜎2 of jth regime follows a
𝑡,(𝑠) (𝑠) 𝑖,(𝑠) 𝑡−𝑖,(𝑠) (𝑠) 𝑡−𝑗,(𝑠) 𝑗𝑡
𝑡 𝑡 𝑡 𝑡 𝑡 𝑡
𝑖=1 𝑗=1 univariateAPGARCHprocessasfollows:
where, 𝜎𝑗𝛿𝑡 =𝑤𝑗+𝛼1𝑗󵄨󵄨󵄨󵄨𝜀𝑡+−1󵄨󵄨󵄨󵄨𝛿+𝛼2𝑗󵄨󵄨󵄨󵄨𝜀𝑡−−1󵄨󵄨󵄨󵄨𝛿+𝛽𝑗𝜎𝑗𝛿,𝑡−1, 𝛿>0, (8)
6 TheScientificWorldJournal
where,𝑤𝑗 > 0,𝛼1𝑗, 𝛼2𝑗,𝛽𝑗 ≥ 0,𝑗 = 1,...,𝑘.Forthepower MS-ARMA-FIAPGARCHderivedisafractionalintegra-
term𝛿=2andfor𝛼1𝑗 =𝛼2𝑗,themodelin(8)reducestoMS- tionaugmentedmodelasfollows:
GARCHmodel.SimilartotheDingetal.[19],theasymmetry,
Iwfhthicehpiassctanlelegdat“ilveevsehraogcekseffheacvte,”diesecpaeprtuimrepdabcty,p𝛼a1𝑗ra=m̸𝛼e2t𝑗e[r1s0a9r]e. (1−𝛽(𝑠𝑡)𝐿)𝜎𝑡𝛿,((𝑠𝑠𝑡)𝑡)
expectedtobe𝛼1𝑗 < 𝛼2𝑗 sothattheleverageeffectbecomes =𝜔+((1−𝛽(𝑠)𝐿)−(1−𝛼(𝑠)𝐿)(1−𝐿)𝑑(𝑠𝑡)) (11)
stronger. 𝑡 𝑡
theAHnaoatsh[e1r09a]ppmrooadcehl,twhhateries tshimeialasyrmtomLetiury[t9e9r]mmsohdaevleias ×(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾(𝑠𝑡)𝜀𝑡−1)𝛿(𝑠𝑡),
differentiatedformas
wherethelagoperatorisdenotedby𝐿,autoregressiveparam-
etersare𝛽(𝑠),and𝛼(𝑠)showsthemovingaverageparameters,
𝜎𝑗𝛿𝑡 =𝑤𝑗+𝛼𝑗(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾𝑗𝜀𝑡−1)𝛿+𝛽𝑗𝜎𝑗𝛿,𝑡−1, 𝛿>0, (9) 𝛿fr(a1)ctio>na0l dd𝑡ieffneorteenstita𝑡htieonopptiamraaml petoewrevratrrieasnsbfoertmweaetinon0, th≤e
𝑑(𝑠) ≤ 1 and allows long memory to be integrated to the
with the restrictions 0 < 𝑤𝑗, 𝛼𝑗, 𝛽𝑗 ≥ 0, 𝛾𝑗 ∈ [−1,1] with mo𝑡del. Regime states (𝑠𝑡) are defined with 𝑚 regimes as
regimes 𝑗 = 1,...,𝑘. The MS-APGARCH model of Haas 𝑖 = 1,...,𝑚. The asymmetry term |𝛾(𝑠)| < 1 ensures that
𝑡
[109] reduces to Ding et al. [19] single regime APGARCH positiveandnegativeinnovationsofthesamesizemayhave
model if 𝑗 = 1. Equation (9) reduces to Liu [99] MS- asymmetric effects on the conditional variance in different
APGARCH specification if 𝛼1𝑗 = 𝛼𝑗(1−𝛾𝑗)𝛿 and 𝛽𝑗 = regimes.
𝛼𝑗𝑠(1+𝛾𝑗)𝛿. ARMSiAm-iFlaIArPtoGAthReCMHSm-AoRdMelAn-eAstPsGseAvRerCaHlmmodoedlesl.,BtyheapMplSy--
The MS-ARMA-GARCH type model specification in ing 𝛿(𝑠) = 2, the model reduces to Markov switching
this study assumes that the conditional mean follows MS- 𝑡
fractionally integrated asymmetric GARCH (MS-ARMA-
ARMA process, whereas the conditional variance follows FIAGARCH); if 𝛿(𝑠) = 2 restriction is applied with 𝛾(𝑠) =
regime switching in the GARCH architecture. Accordingly, 0, themodelreduce𝑡stoMarkovswitchingFIGARCH (M𝑡 S-
MS-ARMA-APGARCHarchitecturenestsseveralmodelsby ARMA-FIGARCH).For𝑑(𝑠) =0,modelreducestotheshort
applying certain restrictions. The MS-ARMA-APGARCH 𝑡
memory version, the MS-ARMA-APGARCH model, if the
mthoedceolnidsitdioenriavlemdebaynmanodviMngS-AfrPomGAMRCS-HA(R𝑙,M𝑚A)cpornodcietisosnianl additionalconstraint𝛿(𝑠𝑡) = 2isapplied,themodelreduces
to MS-Asymmetric GARCH (MS-AGARCH). Lastly, for all
varianceprocessasfollows: the models mentioned above if 𝑖 = 1, all models reduce
to single regime versions of the relevant models, namely,
𝑟 the FIAPGARCH, FIAGARCH, FIGARCH, and AGARCH
𝜎𝑡𝛿,((𝑠𝑠𝑡)𝑡) =𝑤(𝑠𝑡)+∑𝑙=1𝛼𝑙,(𝑠𝑡)(󵄨󵄨󵄨󵄨𝜀𝑡−𝑙󵄨󵄨󵄨󵄨−𝛾𝑙,(𝑠𝑡)𝜀𝑡−𝑙)𝛿(𝑠𝑡) mwiothdetlhse, tchoenirstrrealienvtsan𝑖t=sin1galnedre𝛿g(𝑠im) e=v2a,r𝛾ia(𝑠n)ts=. F0o,rthaetympoicdaell,
(10) reducestosingleregimeFIGARCH𝑡 modelo𝑡fBaillieetal.[17].
𝑞
+ ∑𝛽 𝜎𝛿(𝑠𝑡) , 𝛿 >0, TodifferentiatebetweentheGARCHspecifications,forecast
𝑚=1 𝑚,(𝑠𝑡) 𝑡−𝑚,(𝑠𝑡) (𝑠𝑡) performancecriteriacomparisonsareassumed.
3.NeuralNetworkand
where the regime switches are governed by (𝑠𝑡) and the
parameters are restricted as 𝑤(𝑠) > 0, 𝛼𝑙,(𝑠), 𝛽𝑚,(𝑠) ≥ MS-ARMA-GARCHModels
0 with 𝛾𝑙,(𝑠𝑡) ∈ (−1,1), 𝑙 =𝑡 1,...,𝑟. One𝑡 impo𝑡rtant Inthissectionofthestudy,theMultiLayerPerceptron,Rad-
differenceisthatMS-ARMA-APGARCHmodelin(10)allows
ical Basis Function, and Recurrent Neural Network models
thepowerparameterstovaryacrossregimes.Further,ifthe
thatbelongtotheANNfamilywillbecombinedwithMarkov
followingrestrictionsareapplied,𝑙 = 1,𝑗 = 1,𝛿(𝑠𝑡) = 𝛿,the switching and GARCH models. In this respect, Spezia and
modelreducestothemodelofHaas[109]givenin(9).
Paroli[113]isanotherstudythatmergedtheNeuralNetwork
In applied economics literature, it is shown that many
andMS-ARCHmodels.
financialtimeseriespossesslongmemory,whichcanbefrac-
tionallyintegrated.Fractionalintegrationwillbeintroduced
totheMS-ARMA-APGARCHmodelgivenabove. 3.1.MultilayerPerceptron(MLP)Models
3.1.1. MS-ARMA-GARCH-MLP Model. Artificial Neural
2.3.MS-ARMA-FIAPGARCHModel. AndersenandBoller- Network models have many applications in modeling of
slev [111], Baillie et al. [17], Tse [112], and Ding et al. [19] functional forms in various fields. In economics literature,
providedinterestingapplicationsinwhichtheattentionhad the early studies such as Dutta and Shektar [114], Tom and
been directed on long memory. Long memory could be Kiang [115], Do and Grudinsky [116], Freisleben [51], and
incorporated to the model above by introducing fractional Refenes et al. [55] utilize ANN models to option pricing,
integration in the conditional mean and the conditional realestates,bondratings,andpredictionofbankingfailures
varianceprocesses. amongmany,whereasstudiessuchasKanas[40],Kanasand
TheScientificWorldJournal 7
Yannopoulos[65],andShively[117]appliedANNmodelsto if𝑛𝑗,𝑖transitionprobability𝑃(𝑠𝑡 =𝑖|𝑠𝑡−1 =𝑗)isaccepted;
stock return forecasting, and Donaldson and Kamstra [118]
[𝜀 −𝐸(𝜀)]
proposed hybrid modeling to combine GARCH, GJR, and 𝑧 = 𝑡−𝑑
EGARCHmodelswithANNarchitecture. 𝑡−𝑑 √𝐸(𝜀2) (18)
TheMLP,animportantclassofneuralnetworks,consists
of a set of sensory units that constitute the input layer, one 𝑠 → max{𝑝,𝑞} recursive procedure is started by con-
or more hidden layers, and an output layer. The additional structing 𝑃(𝑧𝑠 = 𝑖 | 𝑧𝑠−1), where 𝜓(𝑧𝑡𝜆ℎ) is of the form
linearinputwhichisconnectedtotheMLPnetworkiscalled 1/(1+exp(−𝑥)),atwice-differentiable,continuousfunction
the Hybrid MLP. Hamilton model can also be considered bounded between [0,1]. The weight vector 𝜉 = 𝑤; 𝜓 = 𝑔
as a nonlinear mixture of autoregressive functions, such as
logisticactivationfunctionandinputvariablesaredefinedas
the multilayer perceptron and thus, the Hamilton model is 𝑧𝑡𝜆ℎ =𝑥𝑖,where𝜆ℎisdefinedasin(16).
called Hybrid MLP-HMC models [119]. Accordingly, in the If 𝑛𝑗,𝑖 transition probability 𝑃(𝑧𝑡 = 𝑖 | 𝑧𝑡−1 = 𝑗) is
HMCmodel,theregimechangesaredominatedbyaMarkov
accepted,
chain without making a priori assumptions in light of the
number of regimes [119]. In fact, Hybrid MLP accepts the 𝑓(𝑦 |𝑥,𝑧 =𝑖)
𝑡 𝑡 𝑡
network inputs to be connected to the output nodes with
weightedconnectionstoformalinearmodelthatisparallel 1 {−(𝑦𝑡−𝑥𝑡󸀠𝜑−∑𝐻𝑗=1𝛽𝑗𝑝(𝑥𝑡󸀠𝛾𝑗))2} (19)
withnonlinearMultilayerPerceptron. = exp{ },
2ℎ
In the study, the MS-ARMA-GARCH-MLP model to √2𝜋ℎ𝑡(𝑖) { 𝑡(𝑗) }
be proposed allows Markov switching type regime changes
both in the conditional mean and conditional variance 𝑠 → max{𝑝,𝑞},recursiveprocedureisstartedbyconstruct-
processes augmented with MLP type neural networks to ing𝑃(𝑧𝑠 =𝑖|𝑧𝑠−1).
achieve improvement in terms of in-sample and out-of-
sampleforecastaccuracy. 3.1.2. MS-ARMA-APGARCH-MLP Model. Asymmetric
The MS-ARMA-GARCH-MLP model is defined of the powerGARCH(APGARCH)modelhasinterestingfeatures.
form: In the construction of the model, the APGARCH structure
𝑟 𝑛 of Ding et al. [19] is followed. The model given in (13)
𝑦 =𝑐 +∑𝜃 𝑦 +𝜀 +∑𝜑 𝜀 , is modified to obtain the Markov switching APGARCH
𝑡 (𝑠) 𝑖,(𝑠) 𝑡−𝑖 𝑡,(𝑠) 𝑗,(𝑠) 𝑡−𝑗,(𝑠) (12)
𝑡 𝑡 𝑡 𝑡 𝑡
𝑖=1 𝑗=1 Multilayer Perceptron (MS-ARMA-APGARCH-MLP)
modeloftheform,
𝑝 𝑞
𝜎𝑡2,(𝑠𝑡) =𝑤(𝑠𝑡)+𝑝∑=1𝛼𝑝,(𝑠𝑡)𝜀𝑡2−𝑝,(𝑠𝑡)+𝑞∑=1𝛽𝑞,(𝑠𝑡)𝜎𝑡−𝑞,(𝑠𝑡) 𝜎𝑡𝛿,(,(𝑠𝑠𝑡𝑡))
(13)
+∑ℎ 𝜉 𝜓(𝜏 ,𝑍 𝜆 ,𝜃 ), =𝑤(𝑠)+ ∑𝑝 𝛼𝑝,(𝑠)(󵄨󵄨󵄨󵄨󵄨𝜀𝑡−𝑝󵄨󵄨󵄨󵄨󵄨−𝛾𝑝,(𝑠)𝜀𝑡−𝑝,(𝑠))𝛿,(𝑠𝑡)
ℎ,(𝑠) ℎ,(𝑠) 𝑡,(𝑠) ℎ,(𝑠) ℎ,(𝑠) 𝑡 𝑡 𝑡 𝑡
𝑡 𝑡 𝑡 𝑡 𝑡 𝑝=1
ℎ=1
𝑞 ℎ
where, the regimes are governed by unobservable Markov +∑𝛽 𝜎𝛿,(𝑠𝑡) +∑𝜉 𝜓(𝜏 ,𝑍 𝜆 ,𝜃 ),
process: 𝑞=1 𝑞,(𝑠𝑡) 𝑡−𝑞,(𝑠𝑡) ℎ=1 ℎ,(𝑠𝑡) ℎ,(𝑠𝑡) 𝑡,(𝑠𝑡) ℎ,(𝑠𝑡) ℎ,(𝑠𝑡)
𝑚 (20)
∑𝜎2 𝑃(𝑆 =𝑖|𝑧 ), 𝑖=1,...𝑚.
𝑡(𝑖) 𝑡 𝑡−1 (14)
where, regimes are governed by unobservable Markov
𝑖=1
process. The model is closed as defining the conditional
In the MLP type neural network, the logistic type sigmoid mean as in (12) and conditional variance of the form
functionisdefinedas equation’s (14)–(19) and (20) to augment the MS-ARMA-
GARCH-MLP model with asymmetric power terms to
𝜓(𝜏 ,𝑍 𝜆 ,𝜃 )
ℎ,(𝑠) 𝑡,(𝑠) ℎ,(𝑠) ℎ,(𝑠) obtain MS-ARMA-APGARCH-MLP. Note that, model
𝑡 𝑡 𝑡 𝑡
nest several specifications. Equation (20) reduces to the
−1
𝑙 ℎ
=[1+exp(−𝜏ℎ,(𝑠𝑡)(∑[∑𝜆ℎ,𝑙,(𝑠𝑡)𝑧𝑡ℎ−𝑙,(𝑠𝑡)+𝜃ℎ,(𝑠𝑡)]))] MterSm-A𝛿RM=A-G2AaRnCdH𝛾-𝑝M,(𝑠L)P=m0o.deSlimiinlarl(y1,3)theifmthodeelpnowesetrs
𝑙=1 ℎ=1 (15) MS-GJR-MLP if 𝛿 = 2𝑡 and 0 ≤ 𝛾𝑝,(𝑠𝑡) ≤ 1 are imposed.
The model may be shown as MSTGARCH-MLP model
(12)𝜆ℎ,𝑑 ∼ uniform [−1,+1] (16) isfin𝛿gle=reg1imanedre0str≤ictio𝛾𝑝n,,(𝑠𝑡𝑠)𝑡 ≤= 𝑠1.=Sim1,ilathrley,qbuyotaepdpmlyiondgelas
reduce to their respective single regime variants, namely,
and𝑃(𝑆𝑡 =𝑖|𝑧𝑡−1),thefilteredprobabilitywiththefollowing the ARMA-APGARCH-MLP, ARMA-GARCH-MLP,
representation, ARMA-NGARCH-MLP, ARMA-GJRGARCH-MLP, and
ARMA-GARCH-MLP models (for further discussion in
(𝑃(𝑆𝑡 =𝑖|𝑧𝑡−1)𝛼𝑓(𝑃(𝜎𝑡−1 |𝑧𝑡−1,𝑠𝑡−1 =1))) (17) NN-GARCHfamilymodels,seeBildiriciandErsin[61]).
8 TheScientificWorldJournal
3.1.3. MS-ARMA-FIAPGARCH-MLP Model. Following the For a typical example, consider a MS-ARMA-
methodologydiscussedintheprevioussection,MS-ARMA- FIAPGARCH-MLPmodelrepresentationwithtworegimes:
APGARCH-MLP model is augmented with neuralnetwork
modelingarchitectureandthataccountsforfractionalinte- (1−𝛽 𝐿)𝜎𝛿(1)
(1) 𝑡,(1)
gration to achieve long memory characteristics to obtain
MS-ARMA-FIAPGARCH-MLP. Following the MS-ARMA- =𝑤 +((1−𝛽 𝐿)−(1−𝜙 𝐿)(1−𝐿)𝑑(1))
(1) (1) (1)
FIAPGARCH represented in (11), the MLP type neural
nreeptwreoserkntaautigomneisnatecdhiMevSed-A:RMA-FIAPGARCH-MLP model ×(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾(1)𝜀𝑡−1)𝛿(1) +∑ℎ 𝜉ℎ,(1)𝜓(𝜏ℎ,(1),𝑍𝑡,(1)𝜆ℎ,(1),𝜃ℎ,(1)),
ℎ=1
(1−𝛽(𝑠𝑡)𝐿)𝜎𝑡𝛿,((𝑠𝑠𝑡𝑡)) (1−𝛽(2)𝐿)𝜎𝑡𝛿,((22))
=𝑤(𝑠)+((1−𝛽(𝑠)𝐿)−(1−𝜙(𝑠)𝐿)(1−𝐿)𝑑(𝑠𝑡)) =𝑤(2)+((1−𝛽(2)𝐿)−(1−𝜙(2)𝐿)(1−𝐿)𝑑(2))
𝑡 𝑡 𝑡
×(󵄨󵄨󵄨󵄨󵄨𝜀𝑡−1,(𝑠𝑡)󵄨󵄨󵄨󵄨󵄨−𝛾(𝑠𝑡)𝜀𝑡−1,(𝑠𝑡))𝛿(𝑠𝑡) (21) ×(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾(2)𝜀𝑡−1)𝛿(2) +∑ℎ 𝜉ℎ,(2)𝜓(𝜏ℎ,(2),𝑍𝑡,(2)𝜆ℎ,(2),𝜃ℎ,(2)).
ℎ=1
ℎ
(22)
+∑𝜉 𝜓(𝜏 ,𝑍 𝜆 ,𝜃 ),
ℎ,(𝑠) ℎ,(𝑠) 𝑡,(𝑠) ℎ,(𝑠) ℎ,(𝑠)
𝑡 𝑡 𝑡 𝑡 𝑡
ℎ=1
Followingthedivisionofregressionspaceintotwosub-
spaceswithMarkovswitching,themodelallowstwodifferent
where, ℎ are neurons defined with sigmoid type logistic asymmetric power terms, 𝛿(1) and 𝛿(2), and two different
functions, 𝑖 = 1,...,𝑚 regime states governed by fractionaldifferentiationparameters,𝑑(1)and𝑑(2);asaresult,
unobservable variable following Markov process. Equation differentlongmemoryandasymmetricpowerstructuresare
(21) defines the MS-ARMA-FIAPGARCH-MLP model, allowedintwodistinguishedregimes.
the fractionally integration variant of the MSAGARCH- ItispossibletoshowthemodelasasingleregimeNN-
MLP model modified with the ANN, and the logistic FIAPGARCHmodelif𝑖=1:
activation function, 𝜓(𝜏ℎ,(𝑠),𝑍𝑡,(𝑠)𝜆ℎ,(𝑠),𝜃ℎ,(𝑠)) defined as
𝑡 𝑡 𝑡 𝑡 (1−𝛽𝐿)𝜎𝛿
in (15). Bildirici and Ersin [61] proposes a class of NN- 𝑡
GARCH models including the NN-APGARCH. Similarly,
=𝜔+((1−𝛽𝐿)−(1−𝜙𝐿)(1−𝐿)𝑑)
the MS-ARMA-FIAPGARCH-MLP model reduces to
(23)
the MS-FIGARCH-MLP model for restrictions on the
rpeodwuecrestetromM𝛿S(-𝑠𝑡F)IN=GA2RaCnHd-𝛾M(𝑠𝑡L)P=mo0d.eFlufortrh𝛾e(r𝑠,)th=e m0oadnedl ×(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾𝑗𝜀𝑡−1)𝛿+∑ℎ 𝜉ℎ𝜓(𝜏ℎ,𝑍𝑡𝜆ℎ,𝜃ℎ).
to the MS-FI-GJRGARCH-MLP model if 𝛿(𝑠)𝑡 = 2 and ℎ=1
𝛾(𝑠) is restricted to be in the range of 0 ≤ 𝛾(𝑡𝑠) ≤ 1. The Further, the model reduces to NN-FIGARCH if 𝑖 = 1 and
mino𝑡addedlirteiodnucteos ttoheM0S-T≤GA𝛾R(𝑠C)H-≤MLP1 mreostdreicltii𝑡ofn𝛿.(𝑠O𝑡)n=th1e 𝛿(𝑠1) =𝛿=2inthefashionofBildiriciandErsin[61]:
𝑡
contrary, if single regime restriction is imposed, models (1−𝛽𝐿)𝜎2
𝑡
discussed above, namely, MS-ARMA-FIAPGARCH-MLP,
MSFIGARCH-NN, MSFIGARCH-NN, MSFINGARCH- =𝜔+((1−𝛽𝐿)−(1−𝜙𝐿)(1−𝐿)𝑑)
MLP, MSFIGJRGARCH-MLP, and MSFITGARCH-MLP (24)
models reduce to NN-FIAPGARCH, NN-FIGARCH, NN- ℎ
FIGARCH, NN-FINGARCH, NN-FIGJRGARCH, and ×(󵄨󵄨󵄨󵄨𝜀𝑡−1󵄨󵄨󵄨󵄨−𝛾𝑗𝜀𝑡−1)2+∑𝜉ℎ𝜓(𝜏ℎ,𝑍𝑡𝜆ℎ,𝜃ℎ).
NN-FITGARCH models, which are single regime neural ℎ=1
network augmented GARCH family models of the form
BildiriciandErsin[61]thatdonotpossessMarkovswitching 3.2.RadialBasisFunctionModel. RadialBasisFunctionsare
type asymmetry (Bildirici and Ersin [61]). The model also one of the most commonly applied neural network models
nests model variants that do not possess long memory that aim at solving the interpolation problem encountered
characteristics. By imposing 𝑑(𝑠) = 0 to the fractional in nonlinear curve fitting.Liu and Zhang [120] utilized the
𝑡
integrationparameterwhichmaytakedifferentvaluesunder Radial Basis Function Neural Networks (RBF) and Markov
𝑖 = 1,2,...,𝑚 different regimes, the model in (21) reduces regime-switching regressionsto divide the regression space
to MS-ARMA-APGARCH-MLP model, the short memory intotwosub-spacestoovercomethedifficultyinestimating
model variant. In addition to the restrictions applied the conditionalvolatility inherent in stock returns. Further,
above, application of 𝑑(𝑠) = 0 results in models without Santosetal.[121]developedaRBF-NN-GARCHmodelthat
𝑡
long memory characteristics: MS-ARMA-FIAPGARCH- benefit from the RBF type neural networks. Liu and Zhang
MLP, MS-ARMA-GARCH-MLP, MS-ARMA-GARCH- [120]combinedRBFneuralnetworkmodelswiththeMarkov
MLP, MSNGARCH-MLP, MS-GJR-GARCH-MLP, and SwitchingmodeltomergeMarkovswitchingNeuralNetwork
MSTGARCH-MLP. modelbasedonRBFmodels.RBFneuralnetworksintheir
TheScientificWorldJournal 9
models are trained to generate both time series forecasts inlightofmodelingaradialfunctionofthedistancebetween
and certainty factors. Accordingly, RBF neural network is theinputsandcalculatedvalueinthehiddenunit.Theoutput
represented as a composition of three layers of nodes; first, unitproducesalinearcombinationofthebasisfunctionsto
theinputlayerthatfeedstheinputdatatoeachofthenodes provideamappingbetweentheinputandoutputvectors:
in the second or hidden layer; the second layer that differs
fdraotmacoluthsteerrnwehuircahlinsectewnoterkresdinattahaptaretaicchulanropdoeinretparnedsehnatssaa 𝜎𝑡𝛿,(,(𝑠𝑠𝑡𝑡)) =𝑤(𝑠𝑡)+𝑝∑𝑝=1𝛼𝑝,(𝑠𝑡)(󵄨󵄨󵄨󵄨󵄨𝜀𝑡−𝑗󵄨󵄨󵄨󵄨󵄨−𝛾𝑝,(𝑠𝑡)𝜀𝑡−𝑝,(𝑠𝑡))𝛿,(𝑠𝑡)
givenradiusandinthethirdlayer,consistingofonenode.
𝑞
3.2.1. MS-ARMA-GARCH-RBF Model. MS-GARCH-RBF +𝑞∑=1𝛽𝑞,(𝑠𝑡)𝜎𝑡𝛿−,(𝑞𝑠,𝑡()𝑠𝑡) (30)
modelisdefinedas
𝜎𝑡2,(𝑠𝑡) = 𝑤(𝑠𝑡)+𝑝∑𝑝=1𝛼𝑝,(𝑠𝑡)𝜀𝑡2−𝑝,(𝑠𝑡) +ℎ∑=ℎ1𝜉ℎ,(𝑠𝑡)𝜙ℎ,(𝑠𝑡)(󵄩󵄩󵄩󵄩󵄩𝑍𝑡,(𝑠𝑡)−𝜇ℎ,(𝑠𝑡)󵄩󵄩󵄩󵄩󵄩),
𝑞 where,𝑖 = 1,...,𝑚regimemodelandregimesaregoverned
+∑𝛽 𝜎
𝑞,(𝑠) 𝑡−𝑞,(𝑠) (25) by unobservable Markov process. Equations (26)–(29) with
𝑡 𝑡
𝑞=1 (30)definetheMS-ARMA-APGARCH-RBFmodel.Similar
totheMS-ARMA-APGARCH-MLPmodel,theMS-ARMA-
+∑ℎ 𝜉ℎ,(𝑠)𝜙ℎ,(𝑠)(󵄩󵄩󵄩󵄩󵄩𝑍𝑡,(𝑠)−𝜇ℎ,(𝑠)󵄩󵄩󵄩󵄩󵄩), APGARCH-RBFmodelnestsseveralmodels.Equation(30)
𝑡 𝑡 𝑡 𝑡 reducestotheMS-ARMA-GARCH-RBFmodelifthepower
ℎ=1
term 𝛿 = 2 and 𝛾𝑝,(𝑠) = 0, to the MSGARCH-RBF model
where 𝑖 = 1,...,𝑚 regimes are governed by unobservable for 𝛾𝑝,(𝑠) = 0, and𝑡to the MSGJRGARCH-RBF model if
Markovprocess: 𝛿 = 2𝑡and 0 ≤ 𝛾𝑝,(𝑠) ≤ 1 restrictions are allowed. The
model may be shown𝑡as MSTGARCH-RBF model if 𝛿 =
𝑚
∑𝜎𝑡2,(𝑠𝑡)𝑃(𝑆𝑡 =𝑖|𝑧𝑡−1). (26) n1aamnedly,0NN≤-A𝛾P𝑝G,(𝑠A𝑡)RC≤H,1N. NFu-rGthAeRr,CsHin,gNleNr-eGgAimReCHm,oNdeNls-,
𝑖=1
NGARCH, NN-GJRGARCH, and NN-TGARCH models,
AGaussianbasisfunctionforthehiddenunitsgivenas𝜙(𝑥) may be obtained if 𝑡 = 1 (for further discussion in NN-
for𝑥=1,2,...,𝑋wheretheactivationfunctionisdefinedas, GARCHfamilymodels,seeBildiriciandErsin[61]).
−󵄩󵄩󵄩󵄩󵄩𝑍𝑡,(𝑠)−𝜇ℎ,(𝑠)󵄩󵄩󵄩󵄩󵄩2 3.2.3. MS-ARMA-FIAPGARCH-RBF Model. MS-
𝜙(ℎ,(𝑠𝑡),𝑍𝑡)=exp( 𝑡2𝜌2 𝑡 ). (27) FIAPGARCH-RBFmodelisdefinedas
𝛿
With 𝑝 defining the width of each function. 𝑍𝑡 is a vector (1−𝛽(𝑠𝑡)𝐿)𝜎𝑡,((𝑠𝑠𝑡)𝑡)
of lagged explanatory variables, 𝛼 + 𝛽 < 1 is essential to
ensure stationarity. Networks of this type can generate any =𝑤(𝑠)+((1−𝛽(𝑠)𝐿)−(1−𝜙(𝑠)𝐿)(1−𝐿)𝑑(𝑠𝑡))
𝑡 𝑡 𝑡
real-valuedoutput,butintheirapplicationswheretheyhave
a priori knowledge of the range of the desired outputs, it ×(󵄨󵄨󵄨󵄨󵄨𝜀𝑡−1,(𝑠)󵄨󵄨󵄨󵄨󵄨−𝛾(𝑠)𝜀𝑡−1,(𝑠))𝛿(𝑠𝑡) (31)
𝑡 𝑡 𝑡
is computationally more efficient to apply some nonlinear
tran𝑃sf(e𝑆r𝑡fu=nct𝑖ion|to𝑧𝑡t−h1e)oiustpthuetsfitolterreefldecptrtohbaatbkinliotywlweditghe.the +∑ℎ 𝜉ℎ,(𝑠𝑡)𝜙ℎ,(𝑠𝑡)(󵄩󵄩󵄩󵄩󵄩𝑍𝑡,(𝑠𝑡)−𝜇ℎ,(𝑠𝑡)󵄩󵄩󵄩󵄩󵄩),
followingrepresentation: ℎ=1
(𝑃(𝑆𝑡 =𝑖|𝑧𝑡−1)𝛼𝑓(𝑃(𝜎𝑡−1 |𝑧𝑡−1,𝑠𝑡−1 =1))). (28) where, ℎ are neurons defined with Gaussian functions.
The MS-ARMA-FIAPGARCH-RBF model is a variant of
If𝑛𝑗,𝑖transitionprobability𝑃(𝑠𝑡 =𝑖|𝑠𝑡−1 =𝑗)isaccepted, the MSAGARCH-RBF model with fractional integration
augmented with ANN architecture. Similarly, the MS-
[𝜀 −𝐸(𝜀)] ARMA-FIAPGARCH-RBFmodelreducestotheMS-ARMA-
𝑧 = 𝑡−𝑑
𝑡−𝑑 (29) FIGARCH-RBF model with restrictions on the power term
√𝐸(𝜀2) 𝛿(𝑠) = 2 and 𝛾(𝑠) = 0. The model nests MSFINGARCH-
RB𝑡Fmodelfor𝛾(𝑡𝑠) = 0,andMSFIGJRGARCH-RBFmodel
𝑠 → max{𝑝,𝑞}recursiveprocedureisstartedbyconstruct- if𝛿(𝑠) = 2and𝛾(𝑠𝑡)variesbetween0 ≤ 𝛾(𝑠) ≤ 1.Further,the
ing𝑃(𝑧𝑠 =𝑖|𝑧𝑠−1). mod𝑡elmaybesho𝑡wnasMSTGARCH-RB𝑡Fmodelif𝛿(𝑠) =1
and0 ≤ 𝛾(𝑠) ≤ 1.Withsingleregimerestriction𝑖 = 1𝑡,dis-
𝑡
3.2.2.MS-ARMA-APGARCH-RBFModel. Radialbasisfunc- cussedmodelsreducetoNN-FIAPGARCH,NN-FIGARCH,
tions are three-layer neural network models with linear NN-FIGARCH, NN-FINGARCH, NN-FIGJRGARCH, and
outputfunctionsandnonlinearactivationfunctionsdefined NN-FITGARCH models, which do not possess Markov
as Gaussian functionsin hidden layer utilizedto the inputs switching type asymmetry. To obtain the model with short
10 TheScientificWorldJournal
imnteemgroartyiocnhapraarcatmereistetircss,sh𝑑o(⋅u)ld=be0irmesptroiscetdionanodnthfreacmtioondaell 𝜎𝑡𝛿,(,(𝑠𝑠𝑡𝑡)) = 𝑤(𝑠𝑡)+𝑝∑𝑝=1𝛼𝑝,(𝑠𝑡)(󵄨󵄨󵄨󵄨󵄨𝜀𝑡−𝑗󵄨󵄨󵄨󵄨󵄨−𝛾𝑝,(𝑠𝑡)𝜀𝑡−𝑝,(𝑠𝑡))𝛿,(𝑠𝑡)
reduces to MSAPGARCH-RBF model, the short memory
model variant. Additionally, by applying 𝑑(⋅) = 0 with 𝑞
tohreyrcehsatrraiccttieornisstidciss:cMusSsFedIAaPbGovAeR,mCHod-RelBsFw,iMthSoGutAlRonCgHm-ReBmF-, +𝑞∑=1𝛽𝑞,(𝑠𝑡)𝜎𝑡𝛿−,(𝑞𝑠,𝑡()𝑠𝑡) (33)
MSGARCH-RBF, MSNGARCH-RBF, MSGJRGARCH-RBF,
ℎ
andMSTGARCH-RBFmodelscouldbeobtained. +∑𝜉 Π(𝜃 𝜒 +𝜃 )
ℎ,(𝑠) 𝑘,ℎ,(𝑠) 𝑡−𝑘,ℎ,(𝑠) 𝑘,ℎ,(𝑠)
𝑡 𝑡 𝑡 𝑡
ℎ=1
3.3. Recurrent Neural Network MS-GARCH Models. The 𝑖 = 1,...,𝑚regimesaregovernedbyunobservableMarkov
RNN model includes the feed-forward system; however, process. 𝜃𝑘,ℎ,(𝑠) is the weights of connection from pre to
it distinguishes itself from standard feed-forward network postsynapticn𝑡odes,Π(𝑥)isalogisticsigmoidfunctionofthe
models in the activation characteristics within the layers. formgivenin(15),𝜒𝑡−𝑘,ℎ,(𝑠)isavariablevectorcorresponding
𝑡
The activations are allowed to provide a feedback to units totheactivationsofpostsynapticnodes,theoutputvectorof
withinthesameorprecedinglayer(s).Thisformsaninternal the hidden units, and 𝜃𝑘,ℎ,(𝑠) are the bias parameters of the
𝑡
memory system that enables a RNN to construct sensitive presynaptic nodes and 𝜉𝑖,(𝑠) are the weights of each hidden
internal representations in response to temporal features unitforℎhiddenneurons,𝑡𝑖 = 1,...,ℎ.Theparametersare
foundwithinadataset. estimatedbyminimizingthesumofthesquared-errorloss:
The Jordan [122] and Elman’s [123] networks are simple min𝜆=∑𝑇𝑡−1[𝜎𝑡 − 𝜎̂𝑡]2.Themodelisestimatedbyrecurrent
recurrent networks to obtain forecasts: Jordan and Elman
back-propagation algorithm and by the recurrent Newton
networks extend the multilayer perceptron with context
algorithm.ByimposingseveralrestrictionssimilartotheMS-
units, which are processing elements (PEs) that remember
ARMA-APGARCH-RBF model, several representations are
pastactivity.Contextunitsprovidethenetworkwiththeabil-
shown under certain restrictions. Equation (33) reduces to
itytoextracttemporalinformationfromthedata.TheRNN MS-ARMA-GARCH-RNN model with 𝛿 = 2 and 𝛾𝑝,(𝑠) =
modelemploysbackpropagation-through-time,anefficient 0, to the MSGARCH-RNN model for 𝛾𝑝,(𝑠) = 0, 𝑡and
gradient-descentlearningalgorithmforrecurrentnetworks. to the MSGJRGARCH-RNN model if 𝛿 =𝑡 2 and 0 ≤
Itwasusedasastandardvariantofcross-validationreferred 𝛾𝑝,(𝑠) ≤ 1 restrictions are imposed. MSTGARCH-RNN
to as the leave-one-out method and as a stopping criterion mod𝑡elisobtainedif𝛿 = 1and0 ≤ 𝛾𝑝,(𝑠) ≤ 1.Inaddition
suitableforestimationproblemswithsparsedataandsoitis 𝑡
to the restrictions above, if the single regime restriction
identifiedtheonsetofoverfittingduringtraining.TheRNN 𝑖 = 1 is implied, the model given in Equation (33) reduces
wasfunctionallyequivalenttoanonlinearregressionmodel
to their single regime variants; namely, the APGARCH-
used for time-series forecasting (Zhang et al. [124]; Binner
RNN, GARCH-RNN, GJRGARCH-RNN, and TGARCH-
etal.[125]).Tinˇoetal.[126]mergedtheRNNandGARCH
RNNmodels,respectively.
models.
3.3.3. MS-ARMA-FIAPGARCH-RNN. Markov Switching
Fractionally Integrated APGARCH Recurrent Neural
3.3.1. MS-ARMA-GARCH-RNN Models. The model is
NetworkModelisdefinedas
definedas
𝛿
(1−𝛽 𝐿)𝜎 (𝑠𝑡)
(𝑠𝑡) 𝑡,(𝑠𝑡)
𝜎2 = 𝑤 + ∑𝑝 𝛼 𝜀2 +∑𝑞 𝛽 𝜎 =𝑤(𝑠)+((1−𝛽(𝑠)𝐿)−(1−𝜙(𝑠)𝐿)(1−𝐿)𝑑(𝑠𝑡))
𝑡,(𝑠𝑡) (𝑠𝑡) 𝑝=1 𝑝,(𝑠𝑡) 𝑡−𝑝,(𝑠𝑡) 𝑞=1 𝑞,(𝑠𝑡) 𝑡−𝑞,(𝑠𝑡) (32) ×(𝑡󵄨󵄨󵄨󵄨󵄨𝜀𝑡−1,(𝑠)󵄨󵄨󵄨󵄨󵄨−𝛾(𝑠𝑡)𝜀𝑡−1,(𝑠))𝛿(𝑠𝑡) 𝑡 (34)
ℎ 𝑡 𝑡 𝑡
+∑𝜉 𝜋 (𝑤 𝜃 +𝜃 ).
ℎ,(𝑠) ℎ,(𝑠) 𝑘,ℎ,(𝑠) 𝑡−𝑘 𝑘,ℎ,(𝑠) ℎ
ℎ=1 𝑡 𝑡 𝑡 𝑡 +∑𝜉ℎ,(𝑠)Π(𝜃𝑘,ℎ,(𝑠)𝜒𝑡−𝑘,ℎ,(𝑠)+𝜃𝑘,ℎ,(𝑠)),
𝑡 𝑡 𝑡 𝑡
ℎ=1
where, ℎ are neurons defined as sigmoid type logistic
Similartothemodelsabove,(32)isshownfor𝑖 = 1,...,𝑚 functions and 𝑖 = 1,...,𝑚 regime states the following
regimes which are governed by unobservable Markov pro- Markovprocess.TheMS-ARMA-FIAPGARCH-RNNmodel
cess.Activationfunctionistakenasthelogisticfunction. is the fractionally integrated variant of the MS-ARMA-
APGARCH-RNN model. The MS-ARMA-FIAPGARCH-
RNN model reduces to the MS-ARMA-FIGARCH-RNN
3.3.2. MS-ARMA-APGARCH-RNN. Markov switching modelwithrestrictionsonthepowerterm𝛿(𝑠) =2and𝛾(𝑠) =
APGARCHRecurrentNeuralNetworkModelisrepresented 0.Further,themodelreducestoMSFINGARC𝑡H-RNNmo𝑡del
as for𝛾(𝑠) =0,totheMSFIGJRGARCH-RNNmodelif𝛿(𝑠) =2
𝑡 𝑡