Controls
Steps
  1. 1. Input Tokens & Config
  2. 2. Token Embeddings
  3. 3. Positional Encodings
  4. 4. Input + Positional Encoding
  5. 5. Encoder Self-Attention
  6. 6. Decoder Target Embeddings
  7. 7. Decoder Positional Encodings
  8. 8. Target + Positional Encoding
  9. 9. Decoder Self-Attention
  10. 10. Decoder-Encoder Attention
  11. 11. Final Output (Softmax)
  12. 5. Head 1: Q, K, V Projection
  13. 6. Head 1: Attention Scores
  14. 7. Head 1: Attention Weights
  15. 8. Head 1: Weighted Values
  16. 9. Head 2: Q, K, V Projection
  17. 10. Head 2: Attention Scores
  18. 11. Head 2: Attention Weights
  19. 12. Head 2: Weighted Values
  20. 13. Concatenate Head Outputs
  21. 14. Final Output (MatMul with W_o)
// Input: "hello world ai"
// Config: dModel=16, nHeads=2, seqLength=4
1. Input Tokens & Config: No data