How LLMs Actually Work

LLM Application Development

TODO

How LLMs Actually Work

• text -> tokenizer -> embedding lookup -> transformer -> unembedding -> softmax -> next token

• Tokenization
  TODO

• Embeddings

• Positional encoding

• Attention

• feed-forward network

• next-token prediction

reference

• How LLMs Actually Work
• LLM Application Development With Python
• [Hands-on Large Language Models: Language Understanding and Generation]