Tag Archives: statistics

Kepler’s Last Stand: Verification by Multiplicity

TNG_LaPalmaFor 3 months a year, the TNG telescope on the island of La Palma turns its high-precision spectrometer (HARPS-N) towards the constellations of Cygnus and Lyra. This is the field of view that NASA’s Kepler space telescope stared at for more than 3 years, detecting thousands of potential new exoplanets using the transit method. There the TNG scans hundreds of Kepler’s potentially planet-holding stars looking for tiny changes in their radial velocity. If detected, this signal will indicate the presence of a real planet, confirming once and for all what Kepler first hinted at many months before. This is the process that, up until now, has been used to definitively find the majority of Kepler’s 211 planets.

New ‘discoveries’ in context

That appeared to change in the blink of an eye this week with the confirmation of 715 new planets using a new catch-all statistical technique. But how did the Kepler team confirm all these new worlds, and can they really be considered real planets?

Without further observations with instruments such as HARPS, Kepler’s 3000 planetary candidates cannot usually be called definite planets. This is because a number of other signals could mimic the transit signal of a star, including tightly bound double-stars that graze one other as they orbit, or unseen dim stars that have binary companions. Alternatively the cameras themselves could be acting up, producing periodic, transit-like signals in the data. Last year a team used simulations of the Kepler data to estimate that around 10% of the candidates were likely to be such false positives.

Kepler Candidates by size

So how can more than 700 worlds be confirmed at once, without any manual work from telescopes on the ground? The answer is through performing statistics on Kepler’s planets. Of a zoo of 190,000 stars observed, Kepler discovered 3000 potential planets, of which 10% are likely to be spurious signals. As a rough estimate then (and the Kepler team go into much more effort than this), the random probability of finding a false positive is 300/190,000, or a rate of only 0.16%.

That number on its own cannot help confirm planets. The trick comes when thinking about Kepler’s hundreds of multiple planet systems. The likelihood of a single-planet system randomly having another false positive also in the data is extremely low. In fact, applying that rough number to the 1000 best single-planet candidates tells us only around 2 of those multiplanet systems should have a spurious planet. Similar calculations can be done for even rarer systems with two false positives, two planets and a false positive, etc.

Possible False Positve Signals

This rate can also be significantly improved by excluding any targets more likely to give these spurious signals. For example, the authors removed more than 350 potential planets from the initial sample for many reasons. Some had instrumental artefacts seen in other stars or had transits close to the limit of detection. Others with V-shaped transits were eliminated as these are more likely to be grazing binary stars. The team also studied the images Kepler took to check for possible transits on a secondary star, eliminating anything where the transit did not in the star’s central position.

Using these cuts, the study narrowed down the search to 851 planets around 340 stars. Applying statistics and using the estimate that 10% of currently detected planets might be false positives, the team found that 849 of the 851 planets were likely to be planets. This corresponds to a certainty of 99.8%,  just greater than 3σ, which in astronomy is usually enough to constitute a detection. This is how “verification by multiplicity” works.

Confirmed Kepler Planets by Size

Of these, 715 are previously un-confirmed worlds. Nearly all are relatively small planets, with radii going from the same as Earth up to that of Neptune. Four of these new planets may also reside in their star’s habitable zone, the region where liquid water could exist on the surface.

As amazing as it would be to nearly double the number of exoplanets overnight, some doubts remain about this method. By eliminating astronomical follow-ups, no extra information can be gleaned. For example, without performing radial velocity measurements, the mass of these planets will never be known. And without other accurate astronomical studies, we cannot accurately determine the nature of the star, and therefore the radius of the planet.

The main difference, though, comes from the impersonal nature of verification by multiplicity. Previous confirmation methods assessed the probability of each candidate being a planet individually. By performing the confirmation in bulk we will know, thanks to the statistics, that at least 2 planets are imposters*. But if exoplanet astronomers can learn to live with that doubt, such planets may well be accepted as confirmed worlds and this simple idea will see the single biggest influx of validated exoplanets in history.

* Here’s another way to compare those statements. Imagine you have two pills. One produces a 0.2% chance of death. The other causes the loss of two fingers (0.2% body mass). By adding these planets to the list of exoplanets, we may well gain a whole new body of worlds, but there will be painful amputations to come in the future.

The two papers, which will be released on March 10th in ApJ, can be found here (Lissauer, 2014) and here (Rowe, 2014).

UPDATE: The new planets are proving reasonably contentious. The exoplanet counter on NASA’s planetquest sits at 1690 , wheras the Paris-based exoplanet.eu remains on 1078. Time will tell whether astronomers accept these as true planets or simply string candidates.

A Planet for Every Star?

Astronomers have now found an astonishing 1000 exoplanets. But that pales in comparison to the 100 billion stars in our galaxy. So how can we say whether planets are the norm? And is it possible to find a star that is definitively a planet-free zone?

The current crop of alien worlds comes from a limited selection of well-studied stars. Rather than try to directly spot what is the equivalent of a fleck of dust in a spotlight, astronomers use changes in the light of the star itself to tease out the signal of a planetary companion.


This can be done in a variety of ways, each of them with their own shortcomings. Often the method of discovery itself means that only a tiny selection of flukily-aligned planets will have the potential of being discovered.

For example, the Kepler spacecraft was staring at over 100,000 stars to try to detect the drop in light as exoplanets crossed their star. However, the probability of the average planet making this crossing is extraordinarily low. A planet orbiting at 1AU (the same distance from its star as Earth) will be found in only 1 in 200 such systems! To put that in perspective; for each Earth-like planet found by Kepler, 199 more stars with planets exactly like our own will have been be tossed aside.


The other common detection technique, known as radial velocity, is marginally less wasteful. This uses the to-and-fro of the star imprinted in the colour variations (or spectra) to find the delicate gravitational tug of a planet. While this works for planets in most orbits, if they happen to circle their star in a face-on orientation, no signal will be received at all. For both cases, this means that even if no planetary signal is detected at all, we can’t definitively say there isn’t one there.

These techniques are also only sensitive to planets larger than a certain size. While the Kepler mission was able to find Earth-sized worlds, similar transit surveys from the ground will only ever be able to find large Gas Giants. Any Mars or Mercury-sized planets will be missed entirely. Radial Velocity is also limited by size, with Neptune or Super-Earth-sized worlds the current limit. These searches are both also bias towards planets close to their stars. To detect worlds at Earth distances is a much trickier prospect than those scraping the surface of their stars.

So many planets will be missed entirely. How can we talk with any certainty about the number of planets in the whole Galaxy?


Well, because the exact problems with these techniques are known, astronomers can estimate how many planets we expect to find. If we know the number of stars studied and the probability of an orbit being perfectly aligned, we can use the number of planets found to estimate the number of planets around all stars.

For example, a study of gravitational lensing by planets showed that on average every star has a planet larger than 5 Earth Masses from 0.5 to 10AU. Similar studies have also been done with Kepler, finding basically the same number: More than one planet bigger than Earth from 0 to 2AU around every star. It should also be noted that these results also only cover a tiny portion of potential planets. Distant Jupiters or low-mass rocky planets were missed completely. So, as our searches become more and more sensitive to small and distant worlds, those numbers can only go up. It’s likely that on average every star in the Milky Way has its own Solar System with multiple planets.

But what about lonely, planet-less worlds? There are certain to be stars without any planetary material wandering the cosmos. For example, those dislodged from triple-star systems, as can happen due to gravitational resonance and scattering, might not hold onto any planetary material. But until we’re able to study a star in perfect detail and definitively say no planets exist, we are forced to stick with what has become the default setting: all stars have planets, and it’s just a matter of time until we find them.