Abstract
We present a motion planning algorithm for a class of uncertain control-affine nonlinear systems which guarantees runtime safety and goal reachability when using high-dimensional sensor measurements (e.g., RGB-D images) and a learned perception module in the feedback control loop. First, given a dataset of states and observations, we train a perception system that seeks to invert a subset of the state from an observation, and estimate an upper bound on the perception error which is valid with high probability in a trusted domain near the data. Next, we use contraction theory to design a stabilizing state feedback controller and a convergent dynamic state observer which uses the learned perception system to update its state estimate. We derive a bound on the trajectory tracking error when this controller is subjected to errors in the dynamics and incorrect state estimates. Finally, we integrate this bound into a sampling-based motion planner, guiding it to return trajectories that can be safely tracked at runtime using sensor data. We demonstrate our approach in simulation on a 4D car, a 6D planar quadrotor, and a 17D manipulation task with RGB(-D) sensor measurements, demonstrating that our method safely and reliably steers the system to the goal, while baselines that fail to consider the trusted domain or state estimation errors can be unsafe.
This research was supported in part by NSF grants IIS-1750489 and IIS-2113401, ONR grants N00014-21-1-2118, N00014-18-1-2501 and N00014-21-1-2431, and a National Defense Science and Engineering Graduate (NDSEG) fellowship.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Agha-mohammadi, A., Chakravorty, S., Amato, N.: FIRM: sampling-based feedback motion-planning under motion uncertainty and imperfect measurements. In: IJRR (2014)
Bahreinian, M., Mitjans, M., Tron, R.: Robust sample-based output-feedback path planning. In: IROS, pp. 5780–5787. IEEE (2021)
Bonnabel, S., Slotine, J.E.: A contraction theory-based analysis of the stability of the deterministic extended kalman filter. TAC 60(2), 565–569 (2015)
Calliess, J.: Conservative decision-making and inference in uncertain dynamical systems (2014)
Chou, G., Ozay, N., Berenson, D.: Model error propagation via learned contraction metrics for safe feedback motion planning of unknown systems. CDC (2021)
Chou, G., Ozay, N., Berenson, D.: Safe output feedback motion planning from images via learned perception modules and contraction theory. CoRR abs/2206.06553 (2022)
Cosner, R., Singletary, A., Taylor, A., Molnár, T., Bouman, K., Ames, A.: Measurement-robust control barrier functions: certainty in safety with uncertainty in state. In: IROS (2021)
Dani, A.P., Chung, S., Hutchinson, S.: Observer design for stochastic nonlinear systems via contraction-based incremental stability. TAC 60(3), 700–714 (2015)
Dawson, C., Lowenkamp, B., Goff, D., Fan, C.: Learning safe, generalizable perception-based hybrid control with certificates. RA-L (2022)
Dean, S., Matni, N., Recht, B., Ye, V.: Robust guarantees for perception-based control. In: L4DC, vol. 120, pp. 350–360. PMLR (2020)
Dean, S., Taylor, A.J., Cosner, R.K., Recht, B., Ames, A.D.: Guaranteeing safety of learned perception modules via measurement-robust control barrier functions. In: CoRL (2020)
Kawano, Y., Hosoe, Y.: Contraction analysis of discrete-time stochastic systems (2021)
Kloss, A., Martius, G., Bohg, J.: How to train your differentiable filter. In: AURO (2021)
Knuth, C., Chou, G., Ozay, N., Berenson, D.: Planning with learned dynamics: Probabilistic guarantees on safety and reachability via lipschitz constants. IEEE RA-L (2021)
Lakshiliikantham, V., Leela, S.: In: Differential and Integral Inequalities - Theory and Applications: Ordinary Differential Equations, vol. 55, pp. 3–44 (1969)
LaValle, S.: Planning Algorithms. Cambridge University Press (2006)
Majumdar, A., Tedrake, R.: Funnel libraries for real-time robust feedback motion planning. IJRR 36(8), 947–982 (2017)
Manchester, I.R., Slotine, J.E.: Output-feedback control of nonlinear systems using control contraction metrics and convex optimization. In: Australian Control Conference (2014)
Manchester, I.R., Slotine, J.E.: Control contraction metrics: convex and intrinsic criteria for nonlinear feedback design. IEEE Trans. Autom. Control. 62(6), 3046–3053 (2017)
Manchester, I.R., Tang, J.Z., Slotine, J.E.: Unifying robot trajectory tracking with control contraction metrics. In: ISRR, vol. 3, pp. 403–418. Springer (2015)
Maybeck, P.S.: Stochastics models, estimation, and control (1979)
Renganathan, V., Shames, I., Summers, T.H.: Towards integrated perception and motion planning with distributionally robust risk constraints. In: IFAC World Congress (2020)
Singh, S., Landry, B., Majumdar, A., Slotine, J.E., Pavone, M.: Robust feedback motion planning via contraction theory (2019)
Sun, D., Jha, S., Fan, C.: Learning certified control using contraction metric. CoRL (2020)
Sunberg, Z.N., Kochenderfer, M.J.: Online algorithms for pomdps with continuous state, action, and observation spaces. In: ICAPS, pp. 259–263. AAAI Press (2018)
Tsukamoto, H., Chung, S.: Learning-based robust motion planning with guaranteed stability: a contraction theory approach. RA-L 6(4), 6164–6171 (2021)
Tsukamoto, H., Chung, S.: Neural contraction metrics for robust estimation and control: a convex optimization approach. IEEE CSL 5(1), 211–216 (2021)
Veer, S., Majumdar, A.: Probably approximately correct vision-based planning using motion primitives. In: CoRL, vol. 155, pp. 1001–1014. PMLR (2020)
Weng, T.W., Zhang, H., Chen, P.Y., Yi, J., Su, D., Gao, Y., Hsieh, C.J., Daniel, L.: Evaluating the robustness of neural networks: an extreme value theory approach. In: ICLR (2018)
Yang, H., Shi, J., Carlone, L.: TEASER: fast and certifiable point cloud registration. T-RO 37(2), 314–333 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chou, G., Ozay, N., Berenson, D. (2023). Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theory. In: LaValle, S.M., O’Kane, J.M., Otte, M., Sadigh, D., Tokekar, P. (eds) Algorithmic Foundations of Robotics XV. WAFR 2022. Springer Proceedings in Advanced Robotics, vol 25. Springer, Cham. https://doi.org/10.1007/978-3-031-21090-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-21090-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21089-1
Online ISBN: 978-3-031-21090-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)Springer Nature Proceedings excluding Computer Science