Model evaluation and improvement

Goal: Test model in simulation, iterate and improve

What I did:
- Wrote a model adapter for the mujoco simulation to evaluate performance
- explored why model was under performing
- tested various training setups
- improved dataloading speed by caching transformed images instead of full episodes

What worked:
- Unfreezing resnet backbone significantly improved converged MSE (0.9 -> 0.6)
- Unfreezing backbone also improved prediction variance, indicating that the model started to learn more than just the mean of the action distribution

Unfrozen model backbone

Frozen Model Backbone

What failed:
- First model iteration didn't learn anything meaningful, it just drove in a straight line when put into simulation
- Second model iteration (unfrozen backbone) was able to learn a bit better, but still didnt accomplish task
- Running out of RAM. VRAM is still ok but will run our fairly fast if model size increases

Key learning:
- If model is unable to meaningfully learn from features, output variance will stay lower than data distribution variance (can be seen in above graphs)
- essentially the model learns to sit around the mean, learning P(X) instead of P(X|Y)
- Could this be specific to the MSE loss function?
- Starting to look like model cannot learn with no temporally correlated features
Next session:
- test new model architectures
- mainly try to change how the data is used (action chunk predictions, some kind temporal features, etc)
- test different loss functions
- Collect cleaner dataset? (only turn clockwise)

Time spent: 3 hours