Count Bayesie's Recommended Books in Probability and Statistics
My book on probability and statistics is a great way to learn more!
Bayesian Statistics the Fun Way!
If you enjoy reading this blog I really think you’ll love my book “Bayesian Statistics the Fun Way” published by No Starch. The book is designed so that anyone can dive in and learn the basics of Bayesian statistics. Even if the math in this blog is sometimes a bit too much for you, all you need to get started is basic high school algebra. The book starts with a tour of probability as logic, the move on to conditional probabilities and Bayes’ theorem, the on to parameter estimation and hypothesis testing. It includes completely reworked posts from this blog and a ton of new content! If you order from No Starch you can get a free ebook with your print copy (or just order to ebook) or you can order on Amazon, or pick it up at your local book store!
Other great books I recommend
A question I often get is "How did you learn all this stuff?" and the honest answer is: reading. This page is a list of books I've read over the years. I’ve placed them roughly in order of mathematical sophistication required to read through them, but there’s no hard requirement here.
Bernoulli’s Fallacy: Statistical Illogic and the Crisis of Modern Science
By Aubrey Clayton
One of the most important parts of understanding any idea, especially mathematical ideas, is to understand the history and development of those ideas. Sadly this is often brushed to the side in many treatment of any mathematical topic.
The field of statistics is extremely lucky to have Aubrey Clayton, who does a remarkable job in this book (and much of his other writing) developing the rich social history that leads to the development the contemporary statistical landscape.
Whenever I see Aubrey’s writing it’s always absolutely pleasure to read and this book is no exception.
By Richard McElreath
So, unlike most of my recommendations, I actually haven’t gotten a chance to read this yet, but it’s absolutely next on my list. I’m currently working through the lectures online and everything so far seems really excellent. Everyone who has read this book has told me it’s amazing and I really think this is the next logical step after “Bayesian Statistics the Fun Way”. The amount of supporting materials that McElreath has on the linked site is phenomenal and I know it has an update coming soon.
Bayesian Modeling and Computation in Python
By Osvaldo Martin, Ravin Kumar and Junpeng Lao
Here’s my official review for this book I sent to CRC: “By far one of the biggest challenges in the practical (and academic) application of Bayesian Statistics is that practitioners need both a strong understanding of the mathematics of Bayesian statistics as well as fairly sophisticated programming ability. This book does a consistently great job of teaching both of these simultaneously…One great example of this is that way in which practical advice, drawing from both academic experience and software engineering experience, is placed throughout the learning process. Pointing out tools to help avoid errors in your model, along with common libraries that make the process easier, really help the reader feel that they are being onboarded by an experienced, kind and helpful team of Bayesian Practitioners. This book is the advanced, practical Bayesian statistics book that is currently missing from my bookshelf.’
Probability Theory: The Logic of Science
By E.T. Jaynes
This is the book on Bayesian analysis. I really recommend getting a strong foundation in probability and statistics before diving in, only because you'll enjoy it that much more. Jaynes doesn't assume that Bayesian analysis is just an evolution of Classical statistics, but rather starts from first principles and builds it up as a form of logic. This is one of the most important books I have read, period. It is also in that category of books that are never truly "finished" because you could easily spend a life time on a single chapter.
Bayesian Data Analysis
By Andrew Gelman, et al.
This is a tremendous work on theoretical statistics if, as Andrew Gelman phrased it, “theoretical statistics was the theory of applied statistics”. This book used to be recommend by anyone doing Bayesian analysis because it was really the only major, comprehensive work on the subject. This book is brilliant, but it is also fairly challenging. Everyone doing Bayesian stats should have a copy of this on their desk. I use mine very frequently. That said, this is not a book you sit and read cover to cover easily. McElreath’s and Martin’s books are better places to get introduce into serious Bayesian stats. However nothing changes this books place as a true classic!