Why isn't the maximum entropy approach to modelling stock return distributions from options prices more well studied

Question

I recently came up with an idea of modeling the distribution of asset returns from the price of options using a maximum entropy approach.  (It should be noted that I'm not really in finance so my grasp of the literature is not good).  Turns out someone more than 20 years ago beat me to that idea.  It's here: https://www.jstor.org/stable/2331391?seq=1#metadata_info_tab_contents and apparently people have been adding on to it for years.  

It doesn't bother me that much that it's been done way before I was thinking about econ, but that it doesn't seem like there is that much empirical work done using this approach.  Maybe I'm wrong, but most of the work seems theoretical.  I talked to a professor of finance (I'm a PhD student), and he wasn't even aware of this line of work, suggesting state of the art was deriving moments from a Black-Scholes equation.  At least if I am not mistaken, this is a 100000x better approach for modeling the distribution of future returns of assets given constraints (i.e. that the expected value of the option equals the price adjusted for time value of money).  The reasoning is that you don't have to assume a normal distribution, in fact, the method can, I think, approximate any distribution given enough constraints.   Second, it also approximates the expected return and doesn't just treat that as an exogenous variable which would be the case if you use a Black-Scholes approach.  

All this being said, I'm curious if there are a reason I'm missing why the Black-Scholes approach is used, or empirical asset pricing researchers don't even bother using an approach like this, where you can get an estimate of the expected return of an asset without waiting for the realization of that return.   Maybe I'm missing a broad swath of empirical literature, but I'd be curious why this doesn't have more than 300 cites.  I'd love to hear from someone what I am missing that makes this work much less important than I think it is.  I am sorry I'm just beside myself right now.   I think Buchen and Kelly, whoever they are, got cheated out of a really good paper (but maybe I'm saying that because I got attached to an idea I came up with myself).

Answer 1

I don't know about the reasons why this particular approach was not followed. I do know, however, quite about about the criticisms one gets in the rational inattention literature, a part of which is also based on Shannon entropy.

One concern people have is that they do not understand the basis for using that particular measure of entropy. It's derived from information theoretic considerations, so why should it be relevant to particular economic settings, where considerations about channel capacity and optimal information transmission play no role.  Why should we use Shannon entropy, rather than any of the half-dozen other entropy measures? At the same time, everyone is intimately familiar with central limit theorems. And while they are, of course, aware that the following is false, they nonetheless often seem to make intuitive assessments as if the central limit theorems implied that all probability distributions are normal.  

Another reason might be the fact that the authors are not very well known. If you decide to build on somebody else's research, you are taking a risk. The higher the chance the research you're building on will be accepted, the higher the chance your own work will be accepted. And that chance is higher for better-known authors.

Comments

This is me still trying to boost the authors work for the empirical asset pricing people out there.  I appreciate the response and I think it is a good one and so I upvoted.

However a few comments:

 You only have a finite number of constraints and an infinite dimensional object to optimize so you have to choose something else to constrain it with.  But why choose the more restrictive constraint.  The maximum entropy distribution with a set mean and set standard deviation is a Gaussian, so if the constraints can imply that, you will get a Gaussian out of the model.   However at the same time to critique the more restrictive Gaussian assumption, we know stock returns don't often behave in a Gaussian manner and that they there is the option smile/ fat tails, so why make a distributional assumption that is more restrictive than you have to.  Meanwhile Shannon entropy is the natural entropy to use because AFAIK, it is the only entropy that basically doesn't when applied to a distribution doesn't allow any other "spurious" constraints to be embedded in it.  Given a set of constraints, it satisfies them and is as natural as possible to any other constraint that could be applied but is not applied.  There is a more full throated defense in the seminal papers by Jaynes (see this wikipedia: https://en.wikipedia.org/wiki/Principle_of_maximum_entropy), that I have not read.  But even if you don't like Shannon entropy, use another entropy if you can get an equation out of it.  Either way, there is no reason to restrict oneself to a Gaussian.  

I do agree with the fact that the authors are not well known probably is a big reason.  However, there have been a decent number of theory papers building on this but I could hardly find one empirical paper.

Answer 2

This is not my area, and I suspect that someone who does work in this area can address maximum entropy more directly.

But, at a very general level, the success of a model or a method depends on the insights that it can express. For most purposes, brownian motions (or brownian motions with jumps) have a lot of convenient modeling properties that are useful for modeling purposes even though we don't believe they are the most accurate model for returns or prices. Heck, many of the most successful asset pricing papers assume time is discrete, utility is exponential, etc. These assumptions work well because they can express interesting, new and empirically-grounded insights. The fact that maximum entropy can approximate any distribution given enough constraints is sort of irrelevant.

I have the sense that twenty years ago, there was more interested than there is today in figuring out how to do the modeling tools needed to do good research. Many of the papers cited by the one you linked, like the Rubinstein and Longstaff papers, were published in top finance journals. By contrast, there is not a single paper in the current issues of RFS, JFE or JF that are as technical as these.

I would guess this means that 1) even if you had thought of this article idea today, I doubt it would gather much interest 2) If you can find an application of this idea that allows you to estimate or model something new and interesting, the fact that someone thought of it 20 years ago won't hurt you.

Comments

Yeah that definitely makes sense.  The main advantage of max ent for economists is not the flexible distribution though I think that's more important than I think the average asset pricing person would, the main advantage that I think would have been recognized by economists is that the expected return and expected volitility are both endogenous to the model, whereas you assume in BS that the return is the risk free rate.  This allows you to study the relationship of mean and variance tradeoffs (or the bias of expected versus actual returns) or gives you just one more paramater you can play around with ie backing out the implied interest rate option investors use instead of assuming the risk free rate.  None of those insights can be gleaned from the BS approach at least from the admittedly little I know about that.

---

Many modern asset pricing papers have endogenous and time varying returns, variance etc.  I'm not sure i understand your comment.

---

I might be mistaken as I dont know the literature, but when talking to a professor on this topic, I got the impression (and I'm really extending what he said) since the predominate way to estimate asset returns density from options prices is to use black Scholes to get the variance.  But the only way to do that, is to assume that the expected return of the distribution is the risk free rate.    That means when estimating a density from option prices, the expected return is always the risk free rate.  Note this does not exclude other methods for estimating the expected return like CAPM for instance.  The problem with using CAPM and other asset pricing models, is that it estimates market expectations of returns from realized returns.  This has a couple of problems, 1. with max ent you have two variables to play with, market expected return and the realized return which are calculated independently, with most asset pricing models market expected return is assumed to be expectation[realized return].  2. related, you cant test if market players are biased and have other behavioral abnormalities as expected return is set to E[realized returns], 3.  Most importantly, when calculating market expected returns = E[realized return], you really only have one realization of the future return i.e. you want to calculate the expected return of Ford stock right now for one year in advance, you find 100 different stocks with exactly the same beta and take the expectation of that (obviously simplyfying) to get your expected return.  The problem is you arent really taking iid draws for Ford stock, you may be able to control for things with a linear regression, but again that's far from actually getting to simulate Ford stock 100 times from now until one year from now.  With max ent, you arent simulating Ford stock 100 times, but using 1000s of peoples choices to buy options and using that to back out the implied expected return from market participants, which is actually even better for calculating what the market expects future returns to be than being able to simulate Ford stock 1000 times, because you dont have to assume market expectations of return = E[return].  Does this makes sense?  I'm not great at writing this just in a comment box and as always I could be wrong.