Evaluating the model in a more realistic sim environment

Goal: Create a more realistic sim environment (prepping for sim2real) and see if model can generalize

What I did:
- create a simulated bedroom environment in mujoco
- created simulated rubiks cubes (for navigation and manipulation task)
-

• tested if model works in new sim environment
• collected 75 episodes of training data on similar task (locate rubiks cube in scene)
• added model metadata to weight files to stop wasting time modifying params in simulation file
• trained model on new dataset with good results (~70-80% task completion rate)
• 

What worked:
- created a room scene fairly easily
- got randomized lighting working
-
-

What failed:
- original green cube task model was not able to generalize to new scene, although this makes sense since it was completely different from the original training data

Key learning:
- got more familiar with mujoco scene building
- when dataset variety is low, generalizability is low

Next session:
- begin sim 2 real testing and data collection
- write model inferencing code for ros2 / hardware interface
- inference on gpu, either write web server or use ros2 directly
- test pure sim policy on real hardware
- set up real world data collection pipeline (ros2 bag post-processing?)
- publish devlog on eryk.ca eventually

Time spent: 5 hours