Evolving notes, images and sounds by Luis Apiolaza

If you are writing a book on Bayesian statistics

This post is somewhat marginal to R in that there are several statistical systems that could be used to tackle the problem. Bayesian statistics is one of those topics that I would like to understand better, much better, in fact. Unfortunately, I struggle to get the time to attend courses on the topic between running my own lectures, research and travel; there are always books, of course.

After we had some strong earthquakes in Christchurch we have had limited access to most part of our physical library (still had full access to all our electronic collection). Last week I had a quick visit to the library and picked up three introductory books: Albert’s Bayesian computation with R, Marin and Robert’s Bayesian core: a practical approach to computational Bayesian statistics and Bolstad’s Understanding computational Bayesian statistics (all links to LibraryThing). My intention was to see if I could use one (or several of them) to start on the topic. What follows are my (probably unfair) comments after reading the first couple of chapters of each book.

In my (highly individual and dubious) opinion Albert’s book is the easiest to read. I was waiting to see the doctor while reading—and actually understanding—some of the concepts. The book is certainly geared towards R users and gradually develops the code necessary to run simple analyses from estimating a proportion to fitting (simple) hierarchical linear models. I’m still reading, which is a compliment.

Marin and Robert’s book is quite different in that uses R as a vehicle (like this blog) but the focus is more on the conceptual side and covers more types of models than Albert’s book. I do not have the probability background for this course (or maybe I did, but it was ages ago); however, the book makes me want to learn/refresh that background. An annoying comment on the book is that it is “self-contained”; well, anything is self-contained if one asks for enough prerequisites! I’m still reading (jumping between Albert’s and this book), and the book has managed to capture my interest.

Finally, Bolstad’s book. How to put this? “It is not you, it is me”. It is much more technical and I do not have the time, nor the patience, to wait until chapter 8 to do something useful (logistic regression). This is going back to the library until an indeterminate future.

If you are now writing a book on the topic I would like to think of the following user case:

  • the reader has little or no exposure to Bayesian statistics, but it has been working for a while with ‘classical’ methods,
  • the reader is self-motivated, but he doesn’t want to spend ages to be able to fit even a simple linear regression,
  • the reader has little background on probability theory, but he is willing to learn some in between learning the tools and to run some analyses,
  • using a statistical system that allows for both classical and Bayesian approaches is a plus.

It is hard for me to be more selfish in this description; you are potentially writing a book for me.

P.S. After publishing this post I remembered that I came across a PDF copy of Doing Bayesian Data Analysis: A Tutorial with R and BUGS by Kruschke. Setting aside the dodginess of the copy, the book looked well-written, started from first principles and had puppies on the cover (!), so I ordered it from Amazon.

P.D. 2011-12-03 23:45 AEST Christian Robert sent me a nice email and wrote a few words on my post. Yes, I’m still plodding along with the book although I’m taking a ten day break while traveling in Australia.

P.D. 2011-11-25 12:25 NZST Here is a list of links to LibraryThing for the books suggested in the comments:


  1. Dave

    I've looked at Albert's and Bolstad's books. You might like Introduction to Applied Bayesian Statistics and Estimation for Social Scientists as an alternative. It gives the math you need to understand basic models, but doesn't go into crazy distributions that you rarely see within social science. Also geared toward R users.

    • Luis

      Thanks Dave. I think there is a bunch of us looking for a book that help us make click! I'll have a look in the library.

  2. Antonio

    I strongly recommend Peter Hoff’s http://www.stat.washington.edu/hoff/book.php as a general starting point. Um abraço, Antonio.

    • Luis

      Obrigado. It seems that I have access to the electronic version through my university library.

  3. John K. Kruschke

    Thanks for posting the link to Doing Bayesian Data Analysis. (And thanks for getting it through legitimate sources instead of stealing a copy! 🙂 I think that the book is currently the most accessible book for learning how to actually do Bayesian analyses with MCMC. By "accessible" I mean for non-statisticians or non-mathematicians. Some of the other books mentioned in the post and the comments are also good, but aimed at more mathematical audiences or with different practical goals (e.g., programming your own MCMC samplers instead of using BUGS). Does Doing Bayesian Data Analysis satisfy all four of your desiderata? The answer is definitely yes for #1, #3, and #4. To get into Bayesian analysis as quickly as possible, take a look at the what Ch. 1 says are the essential sections, and try studying only those first. Hope the book serves you well! And check out the blog (linked with my name). Thanks again!

    • Luis

      Hi John,

      Thanks for the comment and writing such a cool book. Interesting blog, by the way.

  4. Ben Bolker

    Gelman and Hill's book is brilliant; don't know if it's distributed electronically. McCarthy's book is a very gentle introduction for ecologists …

    • Antonio

      I agree about Gelman and Hill's book: quite good, specially with you interested in the multilevel stuff.

    • Luis

      Hi Ben,

      I really like the frequentist side of Gelman and Hill's fantastic book (and I'm into that "multilevel stuff") but find their Bayesian coverage a bit lacking. They just cover a few bits and pieces in 3 chapters out of 25. Anyway, I have a copy of the book in my office and it is getting very well used and with plenty of bookmarks. I should write a review of it here… one day.

  5. Vishal Belsare

    John Kruschke’s book is leisurely and shows ‘mercy’ toward a non-stat/non-math readership. There is also, Bayesian Ideas and Data Analysis by Christensen et al which I think can compliment Kruschke’s book. [ http://amzn.com/1439803544 ]

    There are two different books by Gelman, one is more about regression and multilevel models, the other is ‘Bayesian Data Analysis’ of which a new edition is due. There is also Jeff Gill’s book which would a follow-up after getting through the earlier stuff.

    On a slightly tangential note, I think ‘Principles of Uncertainty’ by Kadane makes for fine studying to grok the Bayesian way in general. [John Kadane has made the PDF available too]

    • Vishal Belsare

      That’s Joseph Kadane, the link to his book is : http://uncertainty.stat.cmu.edu/

    • Luis

      I find Bayesian Data Analysis way too complex for the level I am talking in this post. Kadane's book is also too in depth and takes too long to start with a useful application.

      • Vishal Belsare

        I think, John Kruschke’s makes for a fine non-math read. However, there’s a book tittled ‘Bayesian Ideas & Data Analysis’ which (I think) can be profitably and reasonably covered after getting a grip with Kruschke’s help – if some basic notions of calculus and probability are in place, which I think would be the case for most readers. (This is different from the book ‘Bayesian Data Analysis’ by Gelman et al, which is a more detailed reference.)

        There is a previous book by Bolstad ‘Introduction to Bayesian Statistics’ to which I think his more recent book ‘Understanding Computational Bayesian Statistics’ is a sequel.

        The books you have listed look fine. The one by Scott Lynch is good too, but, I think is too brief to stand alone. To get a handle over things, it might be imperative to use 2-3 books in combination.

        • Luis

          Hi Vishal,

          I concur on the use of several books. I also think that it must be fiendishly difficult for book authors to find the right level to pitch this topic.

  6. Manos Parzakonis

    Hi Luis!

    I have the same "headache" a lot of times since my MSc Thesis was on a Bayesian theme (BMA) and never found myself satisfied with any textbook or reference book. For practicioners I would recommned Lynch's book that @Dave suggested at the top of this thread. It's practical by all means!

    • Luis

      Clearly I'll have to give it a go. Cheers.

  7. mark leeds

    twp other nice books for bayesian explanations are

    1) "bayesian data analysis" by gelman, carlin, stern and rubin.

    2) kind of a level or two higher than albert's book but also very nice is the one by robert and casella "introducing monte carlo methods in R".
    of course, the latter is more R oriented than the first. definitely one needs to understand albert's book before going to robert and casella.

  8. Ankur

    Umm has anyone read Bayesian Statistics and Marketing by Rossi Allenby and McCulloch ? I've been told by some peers to read this one as it's very practical and gives you case studies which are practically faced. I however find trouble with the maths and the material isn't too clear. It's supposed to be 'self-contained' …. I wonder what that means !

    • Luis

      Hi Ankur,

      I had a quick look at Bayesian Statistics and Marketing in Amazon.com. My first impression is that the book seems easy to follow and covers and intriguing (to me) topic: statistics in marketing. I'm a sucker for examples in other research areas, so I consider it worth a look.

  9. Ankur

    Wow so you say it's easy to follow !! I'm shattered 🙁 I'm having such a hard time. Like I said the cases in this book are brilliant ! Now if somebody could just make me understand them that would be wonderful !

    • Luis

      Don't get depressed I found the book easy relative to other books that I've been browsing. 😉

  10. Marcio

    Do you know Frank Lad? He works at the Univ. of Canterbury and wrote a book on Bayesian analysis – Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction. You can look here (www.amazon.com/Operational-Subjective-Statistical-Methods-Philosophical/dp/0471143294).
    Saludos desde Brasil!

    • Luis

      Caro Marcio,

      Thanks for the name and link (it seems the link was messed up in the site, but this one should work. I have met only a subset of the Maths/Stats department—nice and weird people—but I hope to, eventually, met most of them.

      Saludos (from Wollongong, NSW, Australia at the moment),


  11. Bill Bolstad

    Hi Luis,
    My book “Understanding Computational Bayesian Statistics” aims to show reader how computational methods basing inference on a (random) sample from the posterior (even though we only know its shape, not its exact density) has revolutionized statistics. There are lots of choices a user makes in setting up an MCMC model. Some of the choices lead to chains with good convergence properties, some don’t. This book shows the pitfalls that can arise, and how to recognize them from the chain output (and conversely to tell when the chain is performing satisfactorily from the output.) I think you can gain worthwhile insight by examining the many figures in the book. If on the other hand, you just want help on how to set up a WinBUGS model, this is not the book you need.

    • Luis

      Hi Bill,

      Thanks for stopping by. As I pointed out in the post “It is not you, it is me”, as the book was targeting a different reader: one certainly better informed than myself. I was not looking for ways to setup (Win)BUGS or any other software for that matter, but for help on understanding the concepts behind Bayesian inference.

      “Understanding Computational Bayesian Statistics” has some very informative figures, but I (selfishly) need more hand holding in this topic. My way of learning requires longer explanations although without falling into verborrhea.

      Best regards and happy new year,


  12. Bill Bolstad

    Hi Luis,
    Perhaps my first book “Introduction to Bayesian Statistics” would be more useful to you. It covers a similar range of topics as any introductory statistics text, but from the Bayesian perspective. It compares Bayesian inferences with their frequentist analogues and shows the advantages the Bayesian perspective has.

    • Luis

      Hi Bill,

      Thanks for the recommendation. I’ll look it up in the library.


© 2024 Palimpsest

Theme by Anders NorenUp ↑