Table Of Content

Topics in Intelligent Engineering and Informatics 16 Ryszard Klempous Jan Nikodem Péter Zoltán Baranyi Editors Accentuated Innovations in Cognitive Info-Communication Topics in Intelligent Engineering and Informatics Volume 16 SeriesEditors ImreJ.Rudas,JohnvonNeumannFacultyofInformatics,ÓbudaUniversity,Budapest,Hungary AnikóSzakál,ÓbudaUniversity,Budapest,Hungary EditorialBoard IldarBatyrshin ,Mexico JózsefBokor,Hungary BernardDeBaets,Belgium HamidoFujita,Japan ToshioFukuda,Japan FumioHarashima,Japan KaoruHirota,Japan EndrePap,Serbia BogdanM.Wilamowski,USA AdvisoryEditors P.Baranyi,Hungary U.Bodenhofer,Austria G.Fichtinger,Canada R.Fullér,Finland A.Galántai,Hungary L.Hluchý,Slovakia M.O.Jamshidi,USA J.Kelemen,CzechRepublic D.Kocur,Slovakia P.Korondi,Hungary G.Kovács,Hungary L.T.Kóczy,Hungary L.Madarász,Slovakia CH.C.Nguyen,USA E.Petriu,Canada R.-E.Precup,Romania S.Preitl,Romania O.Prostean,Romania V.Puri,Italy G.Y.Sallai,Hungary J.Somló,Hungary M.Takács,Hungary J.Tar,Hungary L.Ungvari,Germany A.R.Várkonyi-Kóczy,Hungary P.Várlaki,Hungary L.Vokorokos,Slovakia Thisbookseriesisdevotedtothepublicationofhigh-levelbooksthatcontributeto topic areas related to intelligent engineering and informatics. This includes advanced textbooks, monographs, state-of-the-art research surveys, as well as edited volumes with coherently integrated and well-balanced contributions within the main subject. The main aim is to provide a unique forum to publish books on mathematical models and computing methods for complex engineering problems thatrequiresomeaspectsofintelligencethatincludelearning,adaptability,improving efficiency and management of uncertain and imprecise information. Intelligent engineering systems try to replicate the fundamental abilities of humans and nature in order to achieve sufficient progress in solving complex problems. In an idealcasemulti-disciplinaryapplicationsofdifferentmodernengineeringfieldscan resultinsynergisticeffects.Informaticsandcomputermodelingaretheunderlying toolsthatplayasignificantroleatanystagesofdevelopingintelligentsystems.Soft computing,asacollectionoftechniquesexploitingapproximationandtolerancefor imprecisionanduncertaintyintraditionallyintractableproblems,hasbecomevery effective and popular, primarily because of the synergy derived from its components. The integration of constituent technologies provides complementary methods that allow developing flexible computing tools and solving complex engineeringproblemsinintelligentways. · · Ryszard Klempous Jan Nikodem Péter Zoltán Baranyi Editors Accentuated Innovations in Cognitive Info-Communication Editors RyszardKlempous JanNikodem DepartmentofControlSystems WrocławUniversityofScience andMechatronics andTechnology WrocławUniversityofScience Wrocław,Poland andTechnology Wrocław,Poland PéterZoltánBaranyi SzéchenyiIstvánUniversity Gyo˝r,Hungary ISSN 2193-9411 ISSN 2193-942X (electronic) TopicsinIntelligentEngineeringandInformatics ISBN 978-3-031-10955-3 ISBN 978-3-031-10956-0 (eBook) https://doi.org/10.1007/978-3-031-10956-0 ©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature SwitzerlandAG2023 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Preface Experienceshowsthatnewemergingtechnologiesdrivenewindustriesandinnova- tion.Severalcurrentandemergingtechnologiesaredrivinginnovation,newindus- triesandapplicationsandmostofthemarecomputational,requireindepthapplica- tionofinformationcommunicationtechnologiesandmathematics.Consideringthe emergence of artificial intelligence, virtual and augmented reality, holography and three-dimensionalvideoandTV,itislogicaltoalsostarttoconceivenewapplications andbusinessesdrivenbythem.Covid-19exposedestablishededucationinstitutions andindustriesandestablishedvideoconferencing,atechnologythatmaturedafew years before the onset of the pandemic. Following in the heels of video confer- encing technologies, cognitive Infocommunications with its attendant components of Virtual Reality (VR) and Augmented Reality (AR) are emerging as innovation driversthatwillleadtonewindustriesandprogressivelydominateseveralindustries including education, medicine, television, space engineering and artificial intelli- genceindustry.ThisbookishenceatimelyexpositionoftheapplicationofVRand hasprovidedfocusonelementsofcognitiveinfocommunicationswitheachchapter contributingasnapshotonaspecificcoginitswheel.Thebookprovidesinvestiga- tionsintotheinternalinformationprocessingmechanismsandprocessesofthebrain and natural intelligence (cognitive informatics), and their engineering applications viaaninterdisciplinaryapproach. Chapter “Methods for Analyzing Cognitive Architecture” articulates methods for analysing human “Cognitive Architecture” by focusing on two approaches in relationtotheanalysisofhumancognitiveabilitiesforisolatingfunctionsrequiredin processingoflanguages.Theauthorsdescribeandexplainperceptionofsymmetryin “humanlanguageprocessingwiththenaturalpredictionsofformallanguagetheory”. Theirsecondapproachisontheattendantsimulationsystems. Chapter“CognitiveResonanceandtheArchitectureIssuesofCognitiveInforma- tionSystems”onthedesignofcognitiveinformationsystemsisalogicalprogres- sion from chapter one. It provides a framework for the development and evolu- tion of Cognitive Information Systems (CIS) by including analysis and design of CIS. A joint design scheme is attempted by joining “the various models for CISs andUnderstanding-basedmanagementsystems(UBMSS)”.Abasisisprovidedfor v vi Preface establishing the properties of CIS and UBMSS which permit the formulation of a practical development method for Cognitive Information and Cognitive Manage- ment Systems (CMSs)”. The Zachman architecture is illustrated in the chapter. In thearchitecture,therelationshipbetweenthenaturalworld,andtheabstractworldis bridgedwithacognitiveinformationsystemthatpunchesthroughavirtualsemantic wall. As a logical closure to Part “Different Aspects of Cognitive Systems” of the book,“anexperimentalclinicalmodelforthestudyoftheembodiedcognitionusing dyspraxia” is provided in Chapter “Dyspraxia: An Experimental Clinical Model fortheStudyoftheEmbodiedCognition”.ItextendsEmbodiedCognitionTheory (ECT), developed in the eighties in the psychological field by George Lakoff to the study of its potential contribution to understanding child development and its potentialapplications”.“TheECTmaintainsthatcognitiveandperceptualprocesses are rooted in the sensory motor skills of the organism; it overcomes the classic cognitivist division between mind and body, and highlights that these two entities areintertwinedinoftenunexpectedways”. It is not unusual to translate elements of biological processes into engineering domainssuchashasbeenpopularisedinartificialneuralnetworks,artificialintelli- genceanddeeplearning.Part“ExploringtheUseandApplicationofVirtualReality” of the book does exactly this by translating the foundations laid in Part “Different AspectsofCognitiveSystems”throughsomeofthemostfundamentalmindboggling innovationsinmoderncomputingandengineeringpracticesinVirtualRealityand AugmentedReality.ThisisbroughttotheforebyBorbalaBerkiinChapter“Role of Presence, Memory and Spatial Ability in a Desktop Virtual Reality” on “The role of presence, memory and spatial ability in a desktop Virtual Reality”. The chapter presents some significant human factors which contribute to the virtual reality experience in desktop virtual reality” systems. Some human factors which contributetothevirtualrealityexperienceindesktopvirtualrealityenvironmentare presented including the notion of presence, perception of presence, the theory of picture superiority effect and the relationship between presence and memory. One of the useful benefits of Chapter “Role of Presence, Memory and Spatial Ability inaDesktopVirtualReality”isitspresentationoftheMaxWherevirtualrealityplat- form,“MaxWhereisadesktopvirtualreality”.Itcanbeusedonasimplecomputer, andthevirtualrealityisdisplayedonasimple2Dmonitor.Inaddition,theplatform supportstheuseofhead-mounteddevices(HMD).ThedesktopVRversionismore popularasitdoesnotrequireanyspecialequipmentandthereisnosimulatorsick- ness.Itcanworkasadesktopvirtualrealityandrequiresneithernavigationspecial devices;classicpointingdevicessuchasmouseortouchpadscanbeusedintheVR environment.The“virtualrealityplatformhasseveraldifferentvirtualspaces,with differentdesignsanddifferentaims”. DwellingonhowtodevelopVRservicesnow,Chapter“ExperiencesandGuide- lines for Developing 3D VR Curricula” shares experiences and guidelines on how to develop VR 3D curricula. It is apparent that more and more modern education institutions and educators will experiment with 3D VR curricula. Hence experi- encessharedinthischapterareinvaluabletonewentrantsintothevirtualeducation Preface vii space.Thechapterisasummaryofaninvestigationintothedevelopmentofeduca- tionmaterials“bylecturersofseveralHungarianuniversitiesfora3DVRplatform calledMaxWhere,intermsofcognitiveeffectiveness,digitalfragmentation,affective collaborativeoutcomesandspatialawareness”.ItcapturesfeaturesoftheMaxWhere VRplatformanditsuseincreationanddeliveryof1250educationmaterialsinover 100subjects.Resultsfromsurveystakenbyeighty-tworespondentsarepresentedin thechapter.OtheressentialinformationinthechapterareonVRLearningResearch LaboratoryandVRCurriculumdevelopmentguideandsupport. AlthoughChapter“Text-BasedSecondLanguageLearningintheThree-Dimen- sional Space” is standalone on its own, its contents however present extension to the contents of Chapter “Experiences and Guidelines for Developing 3D VR Curricula”specificallyinareasnotcoveredbyChapter“ExperiencesandGuidelines for Developing 3D VR Curricula” such as the “three-dimensional virtual library model(3DVLM)basedonthehierarchicalschemeoftheancientLibraryofAlexan- dria”.Thisfocusisexemplaryandanecessaryadditiontowhatisrequiredtoestablish atrulyvirtualinstitutioninmodernterms. Chapter “Interaction Patterns of Spatial Navigation and Smartboard Use in VR Workspaces”,theChapter“Dyspraxia:AnExperimentalClinicalModelfortheStudy oftheEmbodiedCognition”dealingwithVRreportsonanotherapplicationofthe MaxWhereplatformtoSpatialNavigationofVirtualSmartboardsinVirtualReality Spaces. It perfectly complements the previous two chapters on the use of VR in educationcurriculumdevelopment,deliveryandlearnerexperiences. The inter-relationship between artificial intelligence and VR is discussed in Chapter“TowardsRapidPrototypingofDigitalTwinsBasedonHand-HeldVideo” within the specific example of smart manufacturing. It then presents a proof-of- concept application of the YOLOv3 algorithm for dynamic object recognition in hand-held videos. This is followed by an outline of a framework for prototyping digital twins for VR platforms”. Digital twins “is a virtual replica of physical resourcesincludingobjects,processes,people,places,infrastructures,systemsand devicesinaCyberPhysicalSystem(CPS)”.Howtodesigndigitaltwinsandtheirinte- grationtoVRenvironmentshasremainedachallenge.Thechapterthereforeprovides this missing link by developing a framework and a proof-of-concept. Chapter 8 concludesPart“ExploringtheUseandApplicationofVirtualReality”ofthebook. Part“PossiblePracticalApplications”ofthebookisdedicatedtosoftwareapplica- tionsincludingcollaborativesoftwaredesign(Chapter“FindingTreasureintheWild: AFieldStudyofaCollaborativeDesignSoftware”),identificationofhandgestures inspecificdomains(Chapter“IdentifyingtheFunctionofHandGesturesfromTheir Form in Political Speech”), user interface design based on traditional Japanese air fingerdrawingsor“Soragaki”(Chapter“UserInterfaceDesignBasedonTraditional Japanese Air Finger Drawings, “Soragaki””) and a general multi-robot simulation framework(Chapter“RobotCore—AGeneralMulti-robotSimulationFramework”). TheseapplicationsprovideextensiontothetheoreticalframeworkinPart“Different AspectsofCognitiveSystems”asevidenceonwhatishappeninginPart“Exploring theUseandApplicationofVirtualReality”dealingwithpracticalimplementations ofcognitiveinfocommunicationsystemsasevidencedbyvirtualrealitysystemsin viii Preface Part “Exploring the Use and Application of Virtual Reality” and interfaces in Part “PossiblePracticalApplications”. Thebookalthoughderivedfromisolatedresearchoutputsfromvariousresearch laboratories presents a synthesised holistic combination of coherent technologies whichwillbecomemoreandmoreprominentinthecomingdecade.Itisthusauseful teachingandreferencetextbookinVR,robotics,virtualclassroomsandinstitutions, medicine and interface design at both the undergraduate and postgraduate levels. EducatorswillfindPart“ExploringtheUseandApplicationofVirtualReality”of the book extremely timely and useful in transforming current static offerings into immersiveeducationexperiencesforstudentsandeducatorsworldwide.Extension tootherfieldssuchasmedicine,theatreartsandtelevisionareequallyapplicable. Melbourne ProfessorJohnsonIhyehAgbinya October2021 SchoolofInformationTechnologyandEngineering MelbourneInstituteofTechnology Theeditorswouldliketoexpresstheirthankstothereviewersfortheirefforts,the workbywhichallthechaptershaveachievedareasonablelevelandquality.Thelist of reviewers are as follows: Gennaro Cordasco, Antonietta Esposito, Milan Gnja- tovic´, Jozsef Katona, Maria Koutsombogera, Miklós Kuczmann, György Molnár, Benedetta Muzii, Stanislav Ondáš, Justine Reverdy, Cecilia Sik-Lányi, Dávid Sik, MariannaZichar. We would like to thank the preface author Professor Johnson Agbinya, from School of Information Technology and Engineering at the Melbourne Institute of Technology for his solid and substantive work on the evaluation of the presented bookchapters. Special thanks must go to the Rector of the Wrocław University of Science and Technology and to professor Andrzej Kucharski, the Dean of the Faculty of Information and Communication Technology, for their essential and financial support. Theeditorsaregratefultoalltheauthorsfortheirexcellentwork. Finally,wewouldliketothankSpringer-Verlagforthesmoothcooperationinthe publicationofthisvolume. Wrocław,Poland RyszardKlempous Wrocław,Poland JanNikodem Gyo˝r,Hungary PéterZoltánBaranyi November2021 Contents DifferentAspectsofCognitiveSystems MethodsforAnalyzingCognitiveArchitecture ....................... 3 CarlVogelandAnnaEsposito CognitiveResonanceandtheArchitectureIssuesofCognitive InformationSystems ............................................... 29 BálintMolnárandDóraMattyasovszky-Philipp Dyspraxia: An Experimental Clinical Model for the Study oftheEmbodiedCognition ......................................... 57 RaffaeleSperandeo, DanielaCantone, ValeriaCioffi, LuciaLucianaMosca, EnricoMoretto, TeresaLongobardi, YariMirkoAlfano,andNelsonMauroMaldonato ExploringtheUseandApplicationofVirtualReality RoleofPresence,MemoryandSpatialAbilityinaDesktopVirtual Reality ........................................................... 79 BorbálaBerki ExperiencesandGuidelinesforDeveloping3DVRCurricula .......... 99 IldikóHorváth Text-BasedSecondLanguageLearningintheThree-Dimensional Space ............................................................. 125 IstvánKárolyBodaandErzsébetTóth InteractionPatternsofSpatialNavigationandSmartboardUse inVRWorkspaces ................................................. 149 AnnaSudárandÁdámCsapó TowardsRapidPrototypingofDigitalTwinsBasedonHand-Held Video ............................................................. 167 CristinaMonsoneandÁdámB.Csapó ix