In this repo we are going to implement GPT-2, an autoregressive language model that consists only of a transformer decoder.