# Gradient Boosting

Ahh, gradient boosting. In addition to having a totally kickass name, this family of machine learning algorithms is currently among the best known approaches for prediction problems on structured data. This series of posts strives to give a comprehensive understanding of gradient boosting by providing intuitive mathematical explanations, from-scratch implementations of key algorithms, and examples of how to apply modern gradient boosting libraries to solve practical data science problems.

I recommend reading through the series in order, since concepts tend to build on earlier ideas.

### Decision Tree from Scratch

A detailed walkthrough of my from-scratch decision tree implementation in python.

### XGBoost Explained

In-depth explanation and mathematical derivation of the XGBoost algorithm

### XGBoost from Scratch

A walkthrough of my from-scratch python implementation of XGBoost.

### XGBoost for Regression in Python

A step-bystep tutorial on regression with XGBoost in python using sklearn and the xgboost library

### Gradient Boosting Multi-Class Classification from Scratch

How to implement multi-class classification for gradient boosting from scratch in python