First-year students benefit from robotics-based programming exercises by learning how to use sensors to gain information on the (changing) world surrounding the robot, how to model this information using data structures, and how to design algorithms for performing meaningful activities. Robotics-based exercises are naturally experiential and team-based and provide among the most memorable teachable moments of first-year programming courses. We summarize the pedagogical challenges that robotics-based exercises face, even under ideal circumstances, and how a university responded to these challenges over 15 consecutive years. We report on the additional challenges faced in late 2020 at the same university as a result of the COVID pandemic, and how the course staff addressed these challenges using programming language implementation and network tools. The crucial components were (1) a custom-built web-based development environment with collaborative features including a built-in compiler, (2) a portable virtual machine, (3) collaborative editing, (4) open source protocols, and (5) peer-to-peer teleconferencing software. We report on the lessons learnt and how to further improve the resilience of robotics-based programming exercises.