RoPE — Full Rotation Matrix Rotary Position Encoding
RoPE rotates each dimension pair (q2i, q2i+1) by angle pos · θi using a 2×2 rotation matrix — requiring both cos and sin. Toggle between three views of the same underlying data.
View:
Hover mode:
cos(pos · θi)
sin(pos · θi)
Sequence position (pos ↓)
Dimension pair index (i →)
0 (fast)64127 (slow)
0 (fast)64127 (slow)
−1+1
Columns alternate cos, sin, cos, sin… matching how cos_pos and sin_pos are broadcast in the code via repeat(..., rep=2).
Sequence position (pos ↓)
Embedding dimension d (pairs of columns = one rotation)
d=0 cosd=128d=255 sin
−1+1
The raw rotation angle φ = pos · θi in radians. Both cos and sin are derived from this single scalar. Colour = angle magnitude (dark = 0, bright = max).