Autonomous Robotic Pepper Harvesting: Imitation Learning in Unstructured Agricultural Environments

🔍 Overview

Automating tasks in outdoor agricultural fields poses significant challenges due to environmental variability, unstructured terrain, and diverse crop characteristics. We present a robotic system for autonomous pepper harvesting designed to operate in these unprotected, complex settings. Utilizing a custom handheld shear-gripper, we collected 300 demonstrations to train a visuomotor policy, enabling the system to adapt to varying field conditions and crop diversity. We achieved a success rate of 28.95% with a cycle time of 31.71 seconds, comparable to existing systems tested under more controlled conditions like greenhouses. Our system demon- strates the feasibility and effectiveness of leveraging imitation learning for automated harvesting in unstructured agricultural environments. This work aims to advance scalable, automated robotic solutions for agriculture in natural settings.

3. Harvest-in-the-wild!

🎬 Videos

Select a category:

and toggle through videos! (all videos are realtime)

🔧 Hardware

You can access the CAD files and the BILL OF MATERIALS for easy replication and customization.

Our robot deployed in the field

📂 Dataset

We are releasing a dataset of 300 pepper harvesting demonstrations. This dataset includes paired RGB video recordings, providing both external and in-hand camera perspectives. Additionally, we are providing the code for preprocessing these videos to extract observation-action data, which includes RGB images, 6D gripper poses, and gripper actuation.

💻 Code

Coming soon on github...

📄 Paper

@misc{kim2024autonomousroboticpepperharvesting,
  title={Autonomous Robotic Pepper Harvesting: Imitation Learning in Unstructured Agricultural Environments}, 
  author={Chung Hee Kim and Abhisesh Silwal and George Kantor},
  year={2024},
  eprint={2411.09929},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2411.09929}, 
}