Sim-to-real prep

Goal: begin testing sim model performance on real hardware, and set up pipeline for real world data collection

What I did:
- ported the nn_conroller class from the mujoco simulator to a ros2 node connected to the existing hardware interface code
- polished the camera driver to downscale images before publishing to reduce network traffic
- bugfixed some hardware interfacing issues
- tested the sim rubiks cube navigation model with no real world data
- ported the web teleoperation code to a ros2 integrated controller
- Set up real world dataset collection pipeline

What worked:
-
-
- Model inference worked and the full hardware interface works as expected
-
- real world episode data collection working
-

What failed:
- sim data only model did not successfully complete the navigation task
- Due to random softmax sampling, the robot moved a little bit, but pretty much only by random chance
- appears that the camera input is too different from sim to work zero-shot

Key learning:
- Sim to real is not trivial (already knew that!)

Next session:
- Collect real world data and fine tune
- convert all datasets to be rgb by default
- convert dataset collection code in mujoco and ros2
- convert the rubiks cube navigation dataset to rgb
- Match sim camera to be more similar to real life?
- more fisheyed, make similar fov
- do some color shifting of the sim camera data
- maybe implement the "realisimifying" as dataset augmentation, not changes to the dataset collections
- maybe increase the contrast of the real camera data

Time spent: 3.5 hours