Train MNIST from Scratch to 95%+ Accuracy
Build a feedforward neural network that classifies handwritten digits using only NumPy and basic matrix operations. No high-level frameworks allowed for the core model.
Goal: Achieve at least 95% test accuracy on the MNIST dataset with a model you implement from scratch, including forward pass, backpropagation, and gradient descent.
Implement Linear Regression with Gradient Descent
Code linear regression from first principles using only Python and NumPy. Derive the gradient of MSE loss by hand and implement batch gradient descent.
Goal: Fit a linear model to a synthetic or real dataset, visualize the loss curve converging, and compare your solution to the closed-form normal equation.
Build a Word Tokenizer from Scratch
Implement BPE (Byte-Pair Encoding) tokenization from scratch. Handle vocabulary building, merge rules, encoding, and decoding without any tokenization libraries.
Goal: Train a BPE tokenizer on a small text corpus, encode arbitrary strings into token IDs, decode them back perfectly, and compare compression ratios against character-level tokenization.
Classify CIFAR-10 Images with a CNN
Design and train a convolutional neural network to classify 32x32 color images into 10 categories. Use data augmentation and batch normalization to push accuracy above 90%.
Goal: Reach at least 90% test accuracy on CIFAR-10 using a CNN you design from scratch in PyTorch, with proper train/val/test splits and training visualization.
Fine-Tune a Model with LoRA on a Custom Dataset
Apply Low-Rank Adaptation (LoRA) to fine-tune a pre-trained language model on a domain-specific dataset. Compare full fine-tuning vs LoRA in terms of parameter count, memory usage, and downstream performance.
Goal: Fine-tune a 1B+ parameter model using LoRA with less than 1% trainable parameters, achieving comparable performance to full fine-tuning on your evaluation metrics.
Implement Multi-Head Self-Attention from Scratch
Write multi-head self-attention in PyTorch from raw linear layers and matrix multiplications. No nn.MultiheadAttention allowed. Include scaled dot-product attention, causal masking, and proper head splitting.
Goal: Create a working multi-head attention module that produces identical outputs to PyTorch's built-in version when given the same weights, and integrate it into a small transformer block.
Build a Sentiment Analysis Pipeline End-to-End
Create a complete NLP pipeline: data collection, cleaning, tokenization, model training, evaluation, and a simple inference API. Compare at least two approaches (e.g., TF-IDF + logistic regression vs a fine-tuned transformer).
Goal: Deliver a reproducible pipeline that achieves >88% accuracy on a standard sentiment dataset, with proper cross-validation, error analysis, and a REST endpoint for inference.
Deploy an ML Model with FastAPI + Docker
Package a trained model into a production-ready API with FastAPI, containerize it with Docker, add health checks, input validation, and basic load testing.
Goal: Serve a model that handles at least 50 requests/second on a single container, with proper error handling, input validation via Pydantic, and a Dockerfile that builds reproducibly.
Train a GAN to Generate Handwritten Digits
Implement a Generative Adversarial Network from scratch in PyTorch. Train the generator and discriminator on MNIST until the generator produces convincing digit images.
Goal: Generate a grid of 64 fake digit images that are visually indistinguishable from real MNIST samples, with a stable training curve showing neither mode collapse nor discriminator domination.
Build a Transformer (Encoder-Decoder) from Scratch
Implement the full transformer architecture from the 'Attention Is All You Need' paper in PyTorch — positional encoding, multi-head attention, feed-forward layers, residual connections, layer norm, and masked decoding.
Goal: Train your transformer on a small machine translation dataset (e.g., Multi30k EN-DE) and achieve a BLEU score above 25, proving the architecture works end-to-end.
Implement PPO for CartPole from Scratch
Code Proximal Policy Optimization (PPO) from scratch in PyTorch. Implement the clipped surrogate objective, generalized advantage estimation (GAE), and value function baseline.
Goal: Solve CartPole-v1 (500 reward) consistently within 300 episodes, with training curves showing stable policy improvement and proper advantage estimation.
Create Adversarial Examples That Fool Inception
Implement FGSM and PGD adversarial attacks against a pre-trained InceptionV3 model. Visualize the perturbations and measure how accuracy degrades as the attack strength increases.
Goal: Generate adversarial images that cause InceptionV3 to misclassify with >90% success rate while keeping perturbations imperceptible (L-inf norm < 8/255).
Build a Neural Style Transfer App
Implement Gatys et al.'s neural style transfer using a pre-trained VGG-19. Extract content and style representations from intermediate layers and optimize a generated image to match both.
Goal: Produce stylized images that convincingly blend the content of a photograph with the artistic style of a painting, with configurable content/style weight balancing.
Optimize Qwen 2.5-7B Inference Latency by 20% on vLLM
Profile and optimize the inference pipeline for Qwen 2.5-7B running on vLLM. Explore techniques such as KV-cache tuning, custom CUDA kernels, speculative decoding, quantization-aware serving, and batch scheduling optimization — all without degrading output quality.
Goal: Achieve a measurable 20%+ reduction in time-to-first-token and tokens-per-second on a standardized benchmark (e.g., ShareGPT traces), with perplexity degradation under 0.5% on a held-out eval set.
Design and Benchmark a Novel Attention Variant
Propose a modification to standard scaled dot-product attention that improves efficiency, length generalization, or downstream performance. Implement it, train small-scale models, and run rigorous ablations against baselines (standard attention, linear attention, Flash Attention).
Goal: Produce a technical report with wall-clock training comparisons, perplexity curves on at least two datasets, and evidence that your variant offers a meaningful tradeoff improvement over existing approaches.