GPT architecture

Goal: Build and test GPT architecture (not training yet)

What I did:
- Finished attention head design
- finished implemeing full GPT architecture
- Implemented basic tokenizer and model inference system

What worked:
- Good progress on model development
- understanding positional encoding pretty well
-
- Got random output, first AI slop generated!
-

What failed:
- Nothing major

Key learning:
- Lots of pytorch familiarity, working with no ai assistance

Next session:
- reconsider inverse vocabulary mapping system design
- make training loop

Time spent: 4 hours