ebook img

MACHINE VISION SYSTEM FOR ROBOTIC APPLE HARVESTING IN FRUITING WALL PDF

152 Pages·2016·3.46 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview MACHINE VISION SYSTEM FOR ROBOTIC APPLE HARVESTING IN FRUITING WALL

MACHINE VISION SYSTEM FOR ROBOTIC APPLE HARVESTING IN FRUITING WALL ORCHARDS By ABHISESH SILWAL A dissertation submitted in partial fulfillment of the requirement for the degree of DOCTOR OF PHILOSOPHY WASHINGTON STATE UNIVERSITY Department of Biological Systems Engineering DECEMBER 2016 © Copyright by ABHISESH SILWAL, 2016 All Rights Reserved © Copyright by ABHISESH SILWAL, 2016 All Rights Reserved To the Faculty of Washington State University: The members of the Committee appointed to examine the dissertation of ABHISESH SILWAL find it satisfactory and recommend that it be accepted. ________________________________________ Manoj Karkee, Ph.D., Chair ________________________________________ Changki Mo, Ph.D. ________________________________________ Qin Zhang, Ph.D. ii ACKNOWLEDGEMENT I would like to express my sincere gratitude to my advisor Dr. Manoj Karkee for his patience and continuous support during my Ph.D. Without his motivation, mentorship, and immense knowledge, this Ph.D. would not have been possible. Beside my advisor, I would like to thank the rest of my thesis committee Prof. Qin Zhang and Dr. Changki Mo for their insightful comments, encouragements, and timely feedback in my work. My sincere appreciation also goes to Linda Root and Patrick Scharf for their administrative and technical support that was essential during my study at WSU. The mechanical design of the apple harvesting robot is an equally important component of my dissertation. This work was accomplished by my graduate fellow Dr. Joe Davidson. It was a pleasure working with him during laboratory mockup’s and field trials. To my best abilities, I’ve tried to appropriately cite his work on manipulator and end- effector design. I also thank all of my friends at CPAAS who have provided their critical feedback during in-house presentations and seminars. Last but not the least, I would like to thank my wife Pooja, who has constantly motivated me in my purse of this degree. Without her and my parents’ support, none of this and where I stand in life would have been possible. iii MACHINE VISION SYSTEM FOR ROBOTIC APPLE HARVESTING IN FRUITING WALL ORCHARDS Abstract by Abhisesh Silwal, Ph.D. Washington State University December 2016 Chair: Manoj Karkee Mechanization in agriculture is often regarded as one of the greatest human achievements in the twentieth century. These technological advancements have led to significant reduction in human effort in the production of bulk agricultural commodities such as corn and wheat. Specialty crop industry such as fresh fruit market, on the other hand, are still dependent upon manual labor for various production operations such as training, pruning, and harvesting. Among these, tree fruit harvesting of high value crops like apples is the most labor intensive and time sensitive task that requires the right number of farm workers at right time. To increase productivity and reduce dependency on seasonal labor, researchers have proposed automated harvesting systems. Because of highly unstructured orchard environment and variable outdoor conditions, these technologies have achieved only limited successes in the past. No commercial viability has been achieved and every apple destined for fresh market is still handpicked. The iv lack of mechanized harvesting system has the potential to threaten the long-term sustainability of fresh fruit industry in the United States and around the world. This dissertation focuses on the study and evaluation of a machine vision and an integrated robotic system for automated harvesting of apples grown in modern fruiting wall orchards. The machine vision algorithm designed to work in orchard environment accurately detected apples growing individually as well as in heavy clusters under variable natural lighting conditions. A pragmatic approach to harvesting (also called hierarchical approach) showed that 98% of the fruit could be detected with iterative imaging and harvesting of most visible fruits. The integrated robotic system with global camera and custom built seven degrees of freedom manipulator successfully picked 84% of attempted fruit with 6 seconds of average harvest time per fruit. This approach of selective apple harvesting with a global camera system and low-cost manipulator show a huge potential for cost-effective robotic solutions for harvesting fresh market apples. However, several limitations still remain to be addressed for commercialization. v TABLE OF CONTENTS Page ACKNOWLEDGEMENT ............................................................................................................. iii ABSTRACT ................................................................................................................................... iv LIST OF TABLES .......................................................................................................................... x LIST OF FIGURES ....................................................................................................................... xi CHAPTER 1: INTRODUCTION ................................................................................................... 1 1.1 OVERVIEW ......................................................................................................................... 1 1.2 CHALLENGES IN AUTOMATED FRUIT HARVESTING .............................................. 2 1.2.1 Biologically Driven and Highly Unstructured Environment ......................................... 3 1.2.2 Worker Health and Safety .............................................................................................. 3 1.2.3 Time Constraints ............................................................................................................ 4 1.3 IMPACT OF AUTOMATION IN AGRICULTURE ........................................................... 4 1.4 DESIGN SPECIFICATIONS FOR ROBOTIC HARVESTER ........................................... 5 REFERENCES ........................................................................................................................... 7 CHAPTER 2: LITERATURE REVIEW ...................................................................................... 10 2.1 MACHINE VISION FOR ROBOTIC HARVESTING ..................................................... 10 2.1.1 Image Acquisition ........................................................................................................ 10 2.1.2 Image Segmentation and Feature Extraction ............................................................... 12 2.2 MECHANICAL DESIGN FOR ROBOTIC HARVESTING ............................................ 15 2.3 RESEARCH OBJECTIVES ............................................................................................... 18 2.4 ORGANIZATION OF THE DISSERTATION.................................................................. 18 REFERENCES ......................................................................................................................... 20 vi CHAPTER 3: IDENTIFICATION OF RED APPLES IN FIELD ENVIRONMENT WITH OVER THE ROW MACHINE VISION SYSTEM...................................................................... 28 ABSTRACT .............................................................................................................................. 28 3.1 INTRODUCTION .............................................................................................................. 29 3.2 MATERIALS AND METHODS ........................................................................................ 31 3.2.1 Imaging Platform and Data Collection ........................................................................ 31 3.2.2 Image Processing for Apple Identification .................................................................. 33 3.2.2.1 Image Preprocessing ............................................................................................. 33 3.2.2.2 Apple Identification Approach ............................................................................. 35 3.2.3 Object Filtering with Color .......................................................................................... 38 3.3 RESULTS AND DISCUSSION ......................................................................................... 39 3.3.1 Apple Detection using CHT......................................................................................... 39 3.3.2 Apple Detection using Blob Analysis .......................................................................... 40 3.3.3 Accuracy Assessment and Discussion ......................................................................... 41 3.4 CONCLUSION ................................................................................................................... 44 REFERENCES ......................................................................................................................... 46 CHAPTER 4: A HIERARCHICAL APPROACH OF APPLE IDENTIFICATION FOR ROBOTIC HARVESTING .......................................................................................................... 49 ABSTRACT .............................................................................................................................. 49 4.1 INTRODUCTION .............................................................................................................. 50 4.2 DESIGN SPECIFICATIONS, OBJECTIVES, AND GOALS .......................................... 52 4.3 STUDY SITE ...................................................................................................................... 53 4.4 MATERIALS AND METHODS ........................................................................................ 54 vii 4.4.1 Data acquisition and imaging platform ........................................................................ 54 4.4.2 Hierarchical Fruit Identification .................................................................................. 56 4.5 RESULTS AND DISCUSSION ......................................................................................... 60 4.6 SUMMARY AND CONCLUSION ................................................................................... 66 ACKNOWLEDGEMENTS ...................................................................................................... 67 REFERENCES ......................................................................................................................... 68 CHAPTER 5: DESIGN, INTEGRATION, AND FIELD EVALUATION OF A ROBOTIC APPLE HARVESTER .................................................................................................................. 71 ABSTRACT .............................................................................................................................. 71 5.1 INTRODUCTION .............................................................................................................. 72 5.2 PRIOR WORK.................................................................................................................... 76 5.2.1 Fruit Identification ....................................................................................................... 76 5.2.2 Manipulation ................................................................................................................ 78 5.3 SYSTEM REQUIREMENTS ............................................................................................. 80 5.4 CROP ENVIRONMENT AND WORKSPACE MODIFICATIONS ................................ 81 5.5 SYSTEM OVERVIEW ...................................................................................................... 85 5.6 MACHINE VISION SYSTEM .......................................................................................... 87 5.6.1 Global Camera System and Apple Identification ........................................................ 88 5.6.2 Exposure Fusion........................................................................................................... 88 5.6.3 Camera Calibration and Hand–Eye Coordination ....................................................... 89 5.6.4 Fruit Localization ......................................................................................................... 93 5.6.5 Apple Prioritization ...................................................................................................... 94 5.6.6 Uniform Background ................................................................................................... 95 viii 5.7 MECHANICAL DESIGN .................................................................................................. 95 5.7.1 Manipulator .................................................................................................................. 96 5.7.2 End–Effector ................................................................................................................ 97 5.8 EXPERIMENTAL SETUP FOR FIELD EVALUATION ................................................. 98 5.9 RESULTS ......................................................................................................................... 101 5.9.1 Machine Vision System ............................................................................................. 101 5.9.2 Manipulation .............................................................................................................. 103 5.10 DISCUSSION ................................................................................................................. 106 5.11 CONCLUSION ............................................................................................................... 110 ACKNOWLEDGEMENTS .................................................................................................... 111 REFERENCES ....................................................................................................................... 112 CHAPTER 6: OVERALL CONCLUSIONS AND RECOMMENDATIONS .......................... 120 6.1 GENERAL CONCLUSIONS ........................................................................................... 120 6.2: FUTURE DIRECTION ................................................................................................... 123 APPENDIX – C++ and MATLAB Vision Codes ...................................................................... 127 ix

Description:
manual labor for various production operations such as training, pruning, and harvesting. integrated robotic system for automated harvesting of apples grown in Safely guide harvested fruit to a storage container whilst avoiding obstacles and . detect sweet peppers independent of their color.
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.