--- title: README emoji: 🌖 colorFrom: gray colorTo: yellow sdk: static pinned: false --- nanochat students banner Day 1 of Nano Chat

Day 1 of Nano Chat

Getting started with language model training

1 🛠️Environment Setup
Set up your Python environment using uv, create a virtual environment, and install all necessary dependencies for the nanochat project.
View setup instructions →
2 🔪Tokenizer Training
Train a custom BPE tokenizer using Rust bindings on 2 billion characters of data, achieving competitive compression ratios compared to GPT-4's tokenizer.
View tokenizer guide →
3 🥦Pre-training
Download a larger dataset and run distributed training across 8 GPUs using torchrun, with metrics tracked in a shared trackio space.
View pre-training steps →
# nanochat students Welcome to the **nanochat students** organization\! This is a community organization for students following Andrej Karpathy's [nanochat course](https://github.com/karpathy/nanochat). We are learning to build a full-stack LLM implementation from tokenization to web serving, all for under $100. ## What is nanochat? nanochat is a complete implementation of an LLM like ChatGPT in a minimal, hackable codebase. It's designed as the capstone project for the LLM101n course by Eureka Labs, teaching you to build and train your own ChatGPT clone end-to-end. ## What You'll Find Here This organization hosts community-contributed resources to help you learn and succeed with nanochat. You'll find: - notebooks that break down the implementation. - spaces that demo or illustrate the concepts we’re learning. - trained models and checkpoints from the community - relevant curated datasets. ## Getting Help and Sharing Ideas The [Discussions](https://huggingface.co/spaces/nanochat-students/README/discussions) section is where you can ask questions, share your training results and report cards, discuss optimization techniques, and collaborate on experiments. ## Contributing We welcome contributions from all students or experts. Here's how you can help: notebooks, demos, models, and articles: - Join the org, we'll give you write access. - If you make anything, share it in this discussion [thread](https://huggingface.co/spaces/nanochat-students/README/discussions/1) - If you can, help answer questions in [discussions](https://huggingface.co/spaces/nanochat-students/README/discussions) Let's make this a fun, supportive, and efficient community of learners. ## **Resources** - nanochat repo - [karpathy/nanochat](https://github.com/karpathy/nanochat) - Introduction post: ["Introducing nanochat: The best ChatGPT that $100 can buy"](https://github.com/karpathy/nanochat/discussions/1)