Use of Assembly Inspired Instructions in the Allowance of Natural Language Processing in ROS

103 Pages·2017·4.82 MB·English

Use of Assembly Inspired Instructions in the Allowance of Natural
Language Processing in ROS
By
Takondwa Kakusa
Thesis submitted to the Faculty of the
Virginia Polytechnic Institute and State University
in partial fulfillment of the requirements for the degree of
Master of Science
in
Computer Engineering
Michael S. Hsiao, Chair
Cameron D. Patterson
Haibo Zeng
June 19, 2018
Blacksburg, Virginia
Keywords: Natural Language Processing, Robotics, ROS
Copyright 2018, Takondwa Kakusa
Use of Assembly Inspired Instructions in the Allowance of Natural
Language Processing in ROS
by
Takondwa Kakusa
(ABSTRACT)
Natural Language processing is a growing field and widely used in both industrial and and
commercial cases. Though it is difficult to create a natural language system that can
robustly react to and handle every situation it is quite possible to design the system to
react to specific instruction or scenario. The problem with current natural language
systems used in machines, though, is that they are focused on single instructions, working
to complete the instruction given then waiting for the next instruction. In this way they
are not set to respond to possible conditions that are explained to them.
In the system designed and explained in this thesis, the goal is to fix this problem by
introducing a method of adjusting to these conditions. The contributions made in this
thesis are to design a set of instruction types that can be used in order to allow for
conditional statements within natural language instructions. To create a modular system
using ROS in order to allow for more robust communication and integration. Finally, the
goal is to also allow for an interconnection between the written text and derived
instructions that will make the sentence construction more seamless and natural for the
user.
The work in this thesis will be limited in its focus to pertaining to the objective of obstacle
traversal. The ideas and methodology, though, can be seen to extend into future work in
the area.
Use of Assembly Inspired Instructions in the Allowance of Natural
Language Processing in ROS
by
Takondwa Kakusa
(GENERAL AUDIENCE ABSTRACT)
With the growth of natural language processing and the development of artificial
intelligence, it is important to take a look how to best allow these to work together. The
main goal of this project is to find a way of integrating natural language so that it can be
used in order to program a robot and in so doing, develop a method of translating that is
not only efficient but also easy to understand. We have found we can accomplish this by
creating a system that not only creates a direct correlation between the sentence and the
instruction generated for the robot to understand, but also one that is able to break down
complex sentences and paragraphs into multiple different instructions. This allows for a
larger amount of robustness in the system.
.
Table of Contents
1 Introduction 1
1.1 Current State of the Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Thesis Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Literature Review 5
2.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Instruction Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Natural Language Processing / Programming . . . . . . . . . . . . . . . . . 7
2.3.1 Language Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3.2 Processing Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.3 Python and NLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4 Robot Operating System (ROS) . . . . . . . . . . . . . . . . . . . . . . . . . 12
iv
2.4.1 ROS Design Principles . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.2 ROS Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.3 ROS Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.5 ROS and Gazebo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.6 OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Instruction Types 19
3.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Basic vs. Conditional Instructions . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 Base Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.4 GET Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5 SET Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
v
3.6 RUN Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.6.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.6.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.6.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.7 CHECK Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.7.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.7.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.7.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.8 LOOP Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.8.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.8.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.8.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.9 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4 Text Processing Layout 37
4.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 NGRAM Key Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.1 Converting Paragraphs to key terms . . . . . . . . . . . . . . . . . . 39
4.2.2 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
vi
4.3 Instruction Creation and routing . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3.1 Basic Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.2 Check Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.3 Looping Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3.4 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5 ROS Nodal Structure 49
5.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.3 Network Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6 Results 59
6.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.2 Collision and Error Detection . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.3 Basic Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6.4 Complex Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.4.1 Basic Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.4.2 Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.5 Case 1: Object detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.6 Case 2: Maze 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.6.1 Path 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
vii
6.6.2 Path 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.6.3 Path 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.7 Case 3: Maze 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.7.1 Path 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.7.2 Path 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
7 Conclusion and Future Work 82
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Bibliography 85
Appendix A Key-terms Charts 90
viii
List of Figures
2.1 ROS Control: Data Flow of controllers . . . . . . . . . . . . . . . . . . . . . 16
2.2 Integration of Gazebo and the ros control packages . . . . . . . . . . . . . . 17
3.1 Base Design Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2 GET Instruction Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3 GET Structure Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 SET Instruction Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5 SET Structure Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.6 RUN Instruction Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.7 RUN Structure Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.8 CHECK Instruction Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.9 CHECK Structure Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.10 LOOP Instruction Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.11 LOOP Structure Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
ix
4.1 Instruction Processing Diagram . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2 Instruction Creation Example . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3 Check Instruction Creation Example . . . . . . . . . . . . . . . . . . . . . . 44
4.4 Check Instruction Routing Example . . . . . . . . . . . . . . . . . . . . . . . 46
4.5 Check Instruction Routing Example . . . . . . . . . . . . . . . . . . . . . . . 47
5.1 ROS Routing Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.2 Physical car . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.3 Simulated car . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.1 Instruction Tree Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.2 Collision detection example . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6.3 Basic instruction example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.4 Basic instruction tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.5 Complex instruction example 1 . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.6 Complex instruction tree 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.7 Complex instruction example 2 . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.8 Complex instruction tree 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.9 Object Detection Path for World 1 . . . . . . . . . . . . . . . . . . . . . . . 69
6.10 Object Detection Tree for World 1 . . . . . . . . . . . . . . . . . . . . . . . 70
6.11 Object Detection Path 1 for Maze 1 . . . . . . . . . . . . . . . . . . . . . . . 72
x