Sensory Systems for Robotic Applications

330 Pages·2023·11.594 MB·English

by  Ravinder Dahiya, Oliver Ozioko, Gordon Cheng

Sensory Systems for Robotic Applications
Robots have come a long way thanks to advances in sensing and computer vision technologies S Sensory Systems for Robotic
e
and can be found today in healthcare, medicine and industry. Researchers have been looking
n
at providing them with senses such as the ability to see, smell, hear and perceive touch in s
o Applications
order to mimic and interact with humans and their surrounding environments.
r
y
Topics covered in this edited book include various types of sensors used in robotics, sensing
S
schemes (e-skin, tactile skin, e-nose, neuromorphic vision and touch), sensing technologies and y
s
their applications including healthcare, prosthetics, robotics and wearables.
t
e
This book will appeal to researchers, scientists, engineers, and graduate and advanced m
students working in robotics, sensor technologies and electronics, and their applications in s

robotics, haptics, prosthetics, wearable and interactive systems, cognitive engineering, f
o
neuro-engineering, computational neuroscience, medicine and healthcare technologies. r

R
o Edited by
b
About the Editors o Ravinder Dahiya, Oliver Ozioko and Gordon Cheng
t
i
c
Ravinder Dahiya is a professor and leader of the Bendable Electronics and Sustainable
A
Technologies (BEST) Research Group in Electrical and Computer Engineering Department at p
Northeastern University, Boston, USA. p
l
i
c
Oliver Ozioko is a lecturer in electrical and electronic engineering at the University of Derby, a
UK. t
i
o
Gordon Cheng is the chair professor and director of the Institute for Cognitive Systems and is n
s
the coordinator of the Center of Competence Neuro-Engineering, Technical University of
Munich, Germany.
E
d
i
The IET International Series on Sensors t
e
d

Editorial board: Sherali Zeadally, University of Kentucky, USA; Wuqiang Yang, Manchester b
University, UK; Edward Sazonov, Alabama University, USA; Nathan Ida, Akron University, USA; y
Desineni “Subbaram” Naidu, Minnesota Duluth University, USA. D
a
This book series covers advanced research and applications of sensor technologies in the h
fields of Information and Communication Technologies, detection, monitoring, measurement, iy
control and automation, robotics, manufacturing, environment, energy, and transport. a
,

O
z
i
o
k
o

a
n
d

C
h
e
The Institution of Engineering and Technology n
theiet.org g
978-1-84919-948-3
IET CONTROL, ROBOTICS AND SENSORS SERIES 97
Sensory Systems for
Robotic Applications
TheIETInternationalBookSeriesonSensors
IETInternationalBookSeriesonSensors—CallforAuthors
Theuseofsensorshasincreaseddramaticallyinallindustries.Theyarefundamentalinawide
rangeofapplicationsfromcommunicationtomonitoring,remoteoperation,processcontrol,
precisionandsafety,androboticsandautomation.Thesedevelopmentshavebroughtnew
challengessuchasdemandsforrobustnessandreliabilityinnetworks,securityinthe
communicationsinterface,andclosemanagementofenergyconsumption.Thisbookseries
coverstheresearchandapplicationsofsensortechnologiesinthefieldsofICTs,security,
tracking,detection,monitoring,controlandautomation,robotics,machinelearning,smart
technologies,productionandmanufacturing,photonics,environment,energy,andtransport.
BookSeriesEditorialBoard
● Prof.NathanIda,UniversityofAkron,USA
● Prof.EdwardSazonov,UniversityofAlabama,USA
● Prof.Desineni“Subbaram”Naidu,UniversityofMinnesotaDuluth,USA
● Prof.WuqiangYang,UniversityofManchester,UK
● Prof.SheraliZeadally,UniversityofKentucky,USA
Proposalsforcoherentlyintegratedinternationalmultiauthorededitedorcoauthored
handbooksandresearchmonographswillbeconsideredforthisbookseries.Eachproposal
willbereviewedbytheIETBookSeriesEditorialBoardmemberswithadditionalexternal
reviewsfromindependentreviewers.Pleasee-mailyourbookproposalto:
[email protected][email protected].
TheIETInternationalBookSeriesonSensors
IETInternationalBookSeriesonSensors—CallforAuthors
Theuseofsensorshasincreaseddramaticallyinallindustries.Theyarefundamentalinawide Sensory Systems for
rangeofapplicationsfromcommunicationtomonitoring,remoteoperation,processcontrol,
precisionandsafety,androboticsandautomation.Thesedevelopmentshavebroughtnew
challengessuchasdemandsforrobustnessandreliabilityinnetworks,securityinthe
communicationsinterface,andclosemanagementofenergyconsumption.Thisbookseries Robotic Applications
coverstheresearchandapplicationsofsensortechnologiesinthefieldsofICTs,security,
tracking,detection,monitoring,controlandautomation,robotics,machinelearning,smart
technologies,productionandmanufacturing,photonics,environment,energy,andtransport.
BookSeriesEditorialBoard Edited by
● Prof.NathanIda,UniversityofAkron,USA Ravinder Dahiya, Oliver Ozioko and Gordon Cheng
● Prof.EdwardSazonov,UniversityofAlabama,USA
● Prof.Desineni“Subbaram”Naidu,UniversityofMinnesotaDuluth,USA
● Prof.WuqiangYang,UniversityofManchester,UK
● Prof.SheraliZeadally,UniversityofKentucky,USA
Proposalsforcoherentlyintegratedinternationalmultiauthorededitedorcoauthored
handbooksandresearchmonographswillbeconsideredforthisbookseries.Eachproposal
willbereviewedbytheIETBookSeriesEditorialBoardmemberswithadditionalexternal
reviewsfromindependentreviewers.Pleasee-mailyourbookproposalto:
[email protected][email protected].
The Institution of Engineering and Technology
Published by The Institution of Engineering and Technology, London, United Kingdom
The Institution of Engineering and Technology is registered as a Charity in England &
Wales (no. 211014) and Scotland (no. SC038698).
© The Institution of Engineering and Technology 2022
First published 2022
This publication is copyright under the Berne Convention and the Universal Copyright
Convention. All rights reserved. Apart from any fair dealing for the purposes of research or
private study, or criticism or review, as permitted under the Copyright, Designs and Patents
Act 1988, this publication may be reproduced, stored or transmitted, in any form or by
any means, only with the prior permission in writing of the publishers, or in the case of
reprographic reproduction in accordance with the terms of licences issued by the Copyright
Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to
the publisher at the undermentioned address:
The Institution of Engineering and Technology
Futures Place
Kings Way, Stevenage
Herts, SG1 2UA., United Kingdom
www.theiet.org
While the authors and publisher believe that the information and guidance given in this
work are correct, all parties must rely upon their own skill and judgement when making use
of them. Neither the author nor publisher assumes any liability to anyone for any loss or
damage caused by any error or omission in the work, whether such an error or omission is
the result of negligence or any other cause. Any and all such liability is disclaimed.
The moral rights of the author to be identified as author of this work have been asserted by
him in accordance with the Copyright, Designs and Patents Act 1988.
British Library Cataloguing in Publication Data
A catalogue record for this product is available from the British Library
ISBN 978-1-84919-948-3 (hardback)
ISBN 978-1-84919-949-0 (PDF)
Typeset in India by Exeter Premedia Services Private Limited
Printed in the UK by CPI Group (UK) Ltd, Croydon
Cover Image copyrights: Astrid Eckert / TUM / ICS_H-1_!22018_6_jpg
Contents
About the Editors xi
1 Development of tactile sensors for intelligent robotics research 1
Yoshiyuki Ohmura , Akihiko Nagakubo , and Yasuo Kuniyoshi
1.1 Introduction 1
1.2 Developed tactile sensors and implementations 3
1.2.1 Conformable and scalable tactile sensor for 3D-curved surfaces 3
1.2.2 High-density tactile sensor for bare-hand-like sensor gloves 6
1.2.3 Stretchable tactile sensor based on inverse problem analysis 10
1.3 Tactile sensing and robotics: future direction 13
1.3.1 Automaton model 14
1.3.2 State-action model 16
1.3.3 Future direction 19
1.4 Conclusion 20
References 23
2 Developmental soft robotics 27
Luca Scimeca and Fumiya Iida
2.1 Introduction 27
2.2 Bio-inspired soft robotics 29
2.2.1 Soft materials and soft actuation 29
2.2.2 Soft robot control, simulation and learning 32
2.3 Developmental soft robotics 33
2.3.1 Facets of development 33
2.3.2 Soft robotics and developmental time scales 35
2.3.3 Design principles 37
2.3.4 Ontogenetics and adaptivity 41
2.4 Challenges and perspectives 42
2.4.1 Evolutionary robotics 43
2.4.2 Complexity and scalability 44
2.4.3 Learning through the body 44
References 45
vi Sensory systems for robotic applications
3 Three- axis tactile sensor using optical transduction mechanism 55
Masahiro Ohka and Hanafiah Yussof
3.1 Introduction 55
3.2 Design concept of the optical three-axis tactile sensor 58
3.2.1 Basic principle 58
3.2.2 Conical-columnar feeler-type optical three-axis tactile sensor 59
3.2.3 Tracking-centroid-movement-type optical three-axis tactile
sensor 61
3.3 Actual design of the optical three-axis tactile sensor 62
3.3.1 Aluminum-dome type 62
3.3.2 Rubber-dome type 62
3.3.3 Tracking-contact-area-movement type 65
3.4 Applications 67
3.4.1 Tasks achieved by three-axis tactile sensing 67
3.4.2 Picking-up and counting paper 68
3.4.3 Human-robot communication 70
3.5 Conclusion 70
References 71
4 Strain sensors for soft robotic applications 75
Oliver Ozioko and Ravinder Dahiya
4.1 Introduction 75
4.2 Mechanisms for strain sensors 76
4.2.1 Strain sensing based on intrinsic properties of materials and
tunneling effect 76
4.2.2 Disconnection and microcrack propagation mechanism 77
4.3 Classification of strain sensors 78
4.3.1 Piezoresistive strain sensors 78
4.3.2 Capacitive-type strain sensors 81
4.3.3 Triboelectric-type strain sensors 83
4.4 Conclusion 84
References 85
5 Neuromorphic principles for large- scale robot skin 91
Florian Bergner , Emmanuel Dean- Leon , and
Gordon Cheng
5.1 Classical engineering approaches are reaching their limits 91
5.1.1 Motivations for robot skin 91
5.1.2 Robot skin 92
5.1.3 Challenges and limits of robot skin 92
5.2 Biology employs a toolbox full of optimized principles 93
5.2.1 Skin receptors are tuned to sense specific stimulus features 93
5.2.2 Skin receptors transduce stimuli features to binary action
potentials 94
Contents vii
5.2.3 Skin information is encoded by different neural codes 94
5.2.4 Skin information ascends somatotopically ordered 95
5.2.5 Skin information is structured and processed hierarchically 95
5.2.6 The cognitive where 96
5.2.7 The cognitive what 97
5.3 Biological principles are the key to large-scale robot skin 97
5.3.1 Neuromorphic event-driven sensors 97
5.3.2 Neuromorphic information representation in hierarchical
structures 98
5.4 Neuromorphic systems realize biological principles 98
5.4.1 Neuromorphic event-driven vision has been engineered first 98
5.4.2 The neuromorphic AER is a standard for transmitting events 101
5.4.3 The send-on-delta principle allows event-driven transmission
and processing in synchronous systems 104
5.4.4 Neuromorphic event-driven skin is under development 105
5.4.5 Neuromorphic information representations mimic the
primary somatosensory cortex 108
5.4.6 Neuromorphic parallel information streams of the cognitive
where and what 110
5.5 The realization of an event-driven large-scale robot skin system 112
5.5.1 Robot skin system 112
5.5.2 Event-driven reactive skin control 115
5.5.3 The benefits 117
References 117
6 Soft three- axial tactile sensors with integrated electronics for
robot skin 125
Alexander Schmitz , Sophon Somlor , Tito Pradhono Tomo ,
Lorenzo Jamone , Richard Sahala Hartanto , Harris Kristanto ,
Wai Keat Wong , Jinsun Hwang , Alexandre Sarazin , Shuji Hashimoto ,
and Shigeki Sugano
6.1 Introduction 125
6.2 Related work 127
6.2.1 Piezoelectric-based sensors 127
6.2.2 Optical-based sensors 128
6.2.3 Hall-effect-based sensors 128
6.2.4 PSECR-based sensors 130
6.2.5 Piezoresistive-based sensors 130
6.2.6 Capacitive-based sensors 132
6.2.7 MEMS-based sensors 134
6.2.8 Proximity detection 134
6.2.9 Summary of related work 134
6.3 Three-axis capacitive soft skin sensor 134
6.3.1 Concept 134
6.3.2 Implementation 135
viii Sensory systems for robotic applications
6.3.3 Experiments 140
6.3.4 Summary 152
6.4 Three-axis Hall-effect sensors 154
6.4.1 Concept 154
6.4.2 Implementation 154
6.4.3 Experiment 157
6.5 Conclusion 166
References 167
7 A review of tactile sensing in e- skin, wearable device, robotic,
and medical service 173
Jian Hu , Junghwan Back , and Hongbin Liu
7.1 Introduction 174
7.2 Hardware of various tactile sensing technologies 175
7.2.1 Resistive 178
7.2.2 Piezoelectric 178
7.2.3 Capacitive 178
7.2.4 Optical 179
7.2.5 Magnetic field 180
7.2.6 Quantum tunneling composite 180
7.2.7 Triboelectric effect 181
7.2.8 Field-effect transistor 181
7.3 Design criterion and performance index of a tactile sensing system 182
7.4 Applications of tactile sensing technologies 183
7.4.1 Development trend of tactile sensing technologies in e-skin 183
7.4.2 Development trend of tactile sensing technologies in a
wearable device 184
7.4.3 Development trend of tactile sensing technologies in robotic 185
7.4.4 Development trend of tactile sensing technologies in
medical service 186
7.5 Challenges and discussion 187
7.5.1 Standardization of fabrication process 187
7.5.2 Data transmission of high-density tactile sensing elements 188
7.5.3 Fault tolerance and autocalibration 188
7.5.4 Layout of sensing elements on an irregular 3D 189
References 189
8 Neuroengineering approaches for cognitive hearing technology 201
Tobias Reichenbach
8.1 Introduction 201
8.2 General aspects of neurofeedback in a hearing aid 202
8.3 Decoding selective attention to speech from the auditory brainstem
response to the temporal fine structure 204
Contents ix
8.4 Decoding speech comprehension from cortical tracking of speech
features 206
8.5 Enhancing speech comprehension through transcranial electric
stimulation 207
8.6 Summary 209
References 209
9 Mobile robot olfaction state- of- the- art and research challenges 213
Lino Marque , Hugo Magalhães , Rui Baptista , and João Macedo
9.1 Introduction 213
9.2 Odour dispersion 214
9.3 Artificial olfaction 216
9.3.1 Gas sensing 217
9.3.2 Flow sensing 223
9.4 Odour source localisation 225
9.4.1 Searching odours 225
9.4.2 Tracking odour plumes 227
9.4.3 Source declaration 230
9.5 Learning in mobile robot olfaction 231
9.5.1 Source-term estimation 231
9.5.2 Policy search 232
9.6 Open challenges 236
9.6.1 Artificial olfaction 236
9.6.2 Odour source localisation 237
9.6.3 Learning to locate odour sources 238
References 239
10 Vision sensors for robotic perception 249
Shan Luo , Daniel Fernandes Gomes , Jiaqi Jiang , and Guanqun Cao
10.1 Introduction 249
10.2 RGB cameras for robotic perception 252
10.3 Stereo cameras 253
10.4 Event cameras 253
10.4.1 Hardware 253
10.4.2 Applications in robotics 254
10.5 Depth cameras 254
10.6 Vision sensors for other modalities 255
10.6.1 Marker-based sensors 256
10.6.2 Image-based sensors 256
10.7 Conclusions 260
Acknowledgements 261
References 261