LLM Architecture — 3D Interactive Visualization

← Back to ClawProduct Hunt

Architecture Components

Input Tokens

Raw text tokenization
Embedding Layer

Token to vector conversion
Positional Encoding

Sequence position information
Transformer Block 1

First attention layer
→ Self-Attention 1

Query, Key, Value mechanism
→ Feed-Forward 1

Non-linear transformations
Transformer Block 2

Second attention layer
→ Self-Attention 2

Higher-level patterns
→ Feed-Forward 2

Feature refinement
Transformer Block 3

Final attention layer
→ Self-Attention 3

Complex reasoning
→ Feed-Forward 3

Output preparation
Output Layer

Final predictions

Select a Component

Click on any component in the left panel or in the visualization to see detailed information about how it works within the Transformer architecture.

Input Tokens

"The cat sat on the"

The

cat

sat

on

the

Embedding Layer

Tokens become dense vector representations

Positional Encoding

Adding position information to embeddings

Transformer Block 1

Multi-Head Self-Attention

Feed-Forward Network

Add & Norm

Add & Norm

Transformer Block 2

Multi-Head Self-Attention

Feed-Forward Network

Add & Norm

Add & Norm

Transformer Block 3

Multi-Head Self-Attention

Feed-Forward Network

Add & Norm

Add & Norm

Output Layer

Next token predictions

45%

mat

30%

down

15%

floor

8%

ground

2%

table

Speed 1.0x