Now reading: People of PI: Data-driven cosmologist Kendrick Smith

People of PI: Data-driven cosmologist Kendrick Smith

New experiments are generating more data than we can possibly store. Kendrick Smith is one of a new breed of cosmologists, mixing physics, data analysis, statistics, and more, to work out not just what we’re seeing, but what it means.

Kendrick Smith.

Kendrick Smith has not always been a cosmologist. Ask the Perimeter Institute Faculty member how he got interested in physics and he’ll tell you about his first PhD – the one in pure mathematics. Then came a detour out of academia and into software development. And then an interest in physics which he indulged by reading textbooks for fun. Then a second PhD, this one in early universe cosmology.

Once settled in physics, Smith made his name as an expert on the cosmic microwave background, the faint and even glow that covers the whole sky. The CMB, as it’s called, is the light that came through at the first moment that the universe became transparent – just 300,000 years after the big bang itself. It’s the oldest light in the universe.

When it was discovered in 1964, the CMB was the first physical evidence for the big bang. When the COBE satellite began mapping its details in the 1990s, it transformed early universe cosmology from competing sets of compelling stories to a data-driven science. A more powerful satellite, WMAP, was launched in 2001, and for that project, Smith was there. He did some theoretical work, but found himself increasingly pulled toward data analysis.

What’s the distinction? “A theoretical physicist might say, here is a new idea about how the big bang might work,” Smith explains. “Then a data analyst might go and do statistical analysis on maps of the cosmic microwave background, to see if that idea is supported or ruled out by the data.

“In between theory and analysis there is an interesting gap, which is figuring out what to look for. If there’s a physics idea for how the big bang might have worked, it can be very non-obvious how to connect that idea to the data. You have to figure out what statistical procedure should be done, or is most optimal for testing a particular physics idea.”

And who better to bridge the gap than a mathematician turned software developer turned physicist? It’s in building that bridge that Smith excels.

“Most people do one or the other – they are theoretical physicists or data analysts – but I do quite a bit of my work in that gap,” he says. “I do find myself pulled more toward the data side. The experiments generate so much data, and there aren’t enough people to do the analysis.”

Smith’s expertise in data analysis has put him on the leading edge of a tremendous cultural shift in cosmology. What was once a data-starved science now is grappling with data overflow.

One new data source has Smith particularly excited. The CHIME radio telescope – for Canadian Hydrogen Intensity Mapping Experiment – is almost too Canadian to be true: it’s an array of four radio receptors, each the size and shape of a snowboarders’ halfpipe, and set in foothills of the British Columbia’s Okanagan Valley. CHIME is a multi-institutional project, and the first new Canadian telescope in decades.

CHIME telescope
The CHIME telescope in Penticton, BC. (Photo Credit: Peter Klages, University of Toronto.)

CHIME’s unusual strengths arise from its unusual design. Unlike most radio telescopes, CHIME is not a dish that turns to keep one part of the sky in focus. The array of four “half-pipes” is stationary; it will measure over half the sky each day as the Earth turns.

The telescope uses off-the-shelf computers – the same computing devices that power cell phones and gaming consoles and server farms – in new ways.

Smith is particularly excited by one of CHIME’s unique strengths: the ability to detect fast radio bursts, or FRBs. Fast radio bursts are rare and mysterious. “Very occasionally – as in 25 times since 2007 – a bright, brief, isolated radio pulse is observed, coming from – we think – very far outside our galaxy,” he explains.

Physicists have no idea what causes FRBs. “It’s actually a very hard, unsolved problem to explain them,” says Smith. “There are a huge variety of theories. Maybe they have something to do with supernovae; maybe they have something to do with neutron stars. There are more exotic speculations involving new particles or fields in theoretical physics.”

It’s a big qualitative mystery, and in physics, such mysteries often point the way toward major advances. CHIME is tailor-made for such a development. Where the last decade of research have identified 25 known fast radio bursts, once it’s up and running, CHIME should be able to detect 10 a day.

Key to this advance are the software advances made by Smith and his colleagues. “CHIME is really a big computer science problem disguised as a physics problem,” he says. “We have so much data that we can’t even save it all to disk. We need really sophisticated algorithms to do a lot of processing in real time to find these events. There are a lot of sources of contamination, and other technical difficulties. The software has to run really fast.”

The telescope is at what Smith calls “a late stage of commissioning.” Computers are being deployed and turned on. Cables are being run. Bugs are being worked out of the system. The software is being tested, mostly at Perimeter and McGill.

The CHIME telescope should be detecting those fast radio bursts in the fall. When CHIME finally comes online, says Smith, “I would guess we will see many things that we don’t expect.”

Will he be out there when the telescope goes live? The mathematician-turned-software-developer-turned-physicist seems taken aback by the question. After all, he can do pretty much everything he needs to from his office at Perimeter. “I’m not sure it’s strictly necessary,” he says. But on the other hand: “It would be fun, if we found our first FRB right there in BC, and we could drink champagne in front of the telescope.”


CHIME, a radio telescope under construction in British Columbia, expects to find hundreds of “fast radio bursts” and generate a trove of data that may help us understand these mysterious signals in space.

/May 10, 2017

Update (May 19, 2014): There have been recent developments relevant to the article below. For more information, see:
The talk given at Princeton on May 15 by Raphael Flauger (New York University and the Institute for Advanced Study)
The Workshop on Primordial Gravitational Waves and Cosmology at the California Institute of Technology (Caltech)
A less technical article in The Washington Post

/Jun 03, 2014

New research has overturned a widely publicized claim about the beginnings of the universe.

/Apr 22, 2015