SHOPPER: Practical Insights on Grasp Strategies for Mobile Manipulation in the Wild
Paper
Abstract
Mobile manipulation systems have advanced significantly in recent years. However, substantial gaps remain that prevent state-of-the-art platforms from achieving widespread real-world deployment, particularly in reliably grasping items in unstructured environments. To help bridge this gap, we develop SHOPPER, a mobile manipulation robot platform designed to push the boundaries of reliable and generalizable grasp strategies. We develop these grasp strategies and deploy them in a real-world grocery store -- an exceptionally challenging setting chosen for its vast diversity of manipulable items, fixtures, and layouts. In this work, we present our detailed approach to designing general grasp strategies towards picking any item in a real grocery store. Additionally, we provide an in-depth analysis of our latest real-world field test, discussing key findings related to fundamental failure modes over hundreds of distinct pick attempts. Through our detailed analysis, we aim to offer valuable practical insights and identify key grasping challenges, which can guide the robotics community towards pressing open problems in the field. Lastly, we provide a dataset of 1200+ grasp attempts in unseen grocery stores.
The Grocery Store Domain
Grocery stores represent an ideal challenge arena for mobile manipulation: they are unmodified environments designed for human interaction, featuring complex manipulation tasks that humans perform effortlessly but remain deeply challenging for robots. This domain offers several key advantages for advancing robotic manipulation research:
Diverse Grasp Strategies
No single grasping method can handle the full spectrum of items in a grocery store. We develop specialized strategies for different item types, shelf configurations, and spatial constraints—including top-down grasps, side grasps, and extraction maneuvers for densely packed shelves. Full implementation details can be found in our paper.
Real-World Field Testing
We deploy our system in two real, unmodified grocery stores—not controlled laboratory environments. Operating after-hours but without any environmental modifications, our robot autonomously navigates aisles and retrieves items from shelves just as they are arranged for human customers. These field tests represent genuine mobile manipulation challenges in human-designed spaces. Across multiple deployment nights, we collected extensive data on system performance:
Key Failure Modes
Through detailed analysis of our field test data, we identified eight fundamental root causes behind pick failures. By understanding these failure modes, we aim to inform the robotics community about the most pressing challenges in real-world mobile manipulation and guide future research toward solving these open problems.
Dataset
We provide a dataset from three unique field testing events spanning two real-world unseen grocery stores. This dataset covers 1200+ pick attempts, labeled as success or failure, over 800+ unique items.
The grasp data includes:
- Robot base and arm actions
- 3D perception from both head and chassis stereo cameras
- Navigation data between items in the grocery stores
We hope this dataset can provide helpful data for further analysis or training (for pre-training of robot foundational models, learning from failure data, etc.).
Citation
@inproceedings{huang2026shopper,
title={SHOPPER: Practical Insights on Grasp Strategies for Mobile Manipulation in the Wild},
author={Huang, Isabella and Cheng, Richard and Kim, Sangwoon and Kruse, Dan and Chen, Carolyn and Kaul, Lukas and Hancock, JC and Harikumar, Shanmuga and Tjersland, Mark and Borders, James and Helmick, Dan},
booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
year={2026}
}