SoylentNews is people
SoylentNews
Nathan Myhrvold: 'Nasa doesn't want to admit it's wrong about asteroids'
Nathan Myhrvold is the former chief technology officer of Microsoft, founder of the controversial patent asset company Intellectual Ventures and the main author of the six-volume, 2,300-page Modernist Cuisine cookbook, which explores the science of cooking. Currently, he is taking on Nasa over its measurement of asteroid sizes.
For the past couple of years, you've been fighting with Nasa about its analysis of near-Earth asteroid size. You've just published a 33-page scientific paper [open, DOI: 10.1016/j.icarus.2018.05.004] [DX] criticising the methods used by its Neowise project team to estimate the size and other properties of approximately 164,000 asteroids. You have also published a long blog post explaining the problem. Where did Nasa go wrong and is it over or underestimating size?
Nasa's Wise space telescope [Wide-field Infrared Survey Explorer] measured the asteroids in four different wavelengths in the infrared. My main beef is with how they analysed that data. What I think happened is they made some poor choices of statistical methods. Then, to cover that up, they didn't publish a lot of the information that would help someone else replicate it. I'm afraid they have both over- and underestimated. The effect changes depending on the size of the asteroid and what it's made of. The studies were advertised as being accurate to plus or minus 10%. In fact, it is more like 30-35%. That's if you look overall. If you look at specific subsets some of them are off by more than 100%. It's kind of a mess.
[...] Nasa's reported response has been to stand by the data and the analysis performed by the Neowise team. Can we trust Nasa after this?
They need to have an independent investigation of these results. When my preprint paper came out in 2016, they said: "You shouldn't believe it because it's not peer-reviewed." Well, now it has been peer reviewed. How Nasa handles it at this stage will be very telling. People have suggested to me the reason Nasa doesn't want to admit that anything is wrong with the data is that they're afraid it would hurt the chances of Neocam, an approximately $500m (£380m) telescope to find asteroids that might hit Earth proposed by the same group who did the Neowise analysis.
Previously: Former Microsoft Chief Technologist Criticizes NASA
(Score: 0) by Anonymous Coward on Monday June 25 2018, @01:43PM (6 children)
He can make his own databse with asteroid sizes, and then argue why his is better than NASA's. Otherwise shut up. Why is this guy getting so much press? Every analysis of complex and messy scientific data like this one will have it's flaws.
(Score: 0) by Anonymous Coward on Monday June 25 2018, @01:49PM (2 children)
He did collect all the data into his own database and released his own analysis. What exactly do you think is missing?
(Score: 1, Informative) by Anonymous Coward on Monday June 25 2018, @03:50PM (1 child)
Note that I have no affiliation with NASA or Myhrvold, but this story has been bugging me.
I see the Mainzer team's papers with a bunch figures and text that describe how to relate WISE images to real asteroid diameters, which is a tricky process. I see Myhrvold spending all of his ink complaining that they didn't do it right.
One complaint seems to be that a table they published contains D values (diameters) from radar observations rather than from their WISE images. Presumably, the NASA people wanted to but the best available diameter in the table (not unreasonable). Apparently they should have been more clear about the source of D, or kept one column for WISE-based D and one for radar-based D. Oh well, the perfect is the enemy of the good.
Another complaint is that the "fits miss all the data points" in some cases, and Myhrvold equates this to data fabrication. If a large part of the database were based on garbage fits, that would be a reasonable criticism. But I don't see any evidence of that. The NASA analysis does include error bars for each asteroid, and a lousy fit will show up as a big error bar.
(Score: 0) by Anonymous Coward on Monday June 25 2018, @07:17PM
whats this have to do with "put up or shut up"?
(Score: 4, Insightful) by ElizabethGreene on Monday June 25 2018, @02:03PM (2 children)
His argument is that they've done bad science by producing unreproducible data. It's bad science to say "Here's a database of asteroid sizes." Good science says "Here's a database of asteroid sizes, the data we used to determine the sizes, the algorithm we processed that data with, the data we calibrated that algorithm with, the parameters we chose to make our algorithm come up with those answers, what our guesstimated error bars are, and (most importantly) where we know the model is wrong." His accusation is that the data, corrections, and algorithm that are publicly available do not produce the results they published.
He also accuses them of making backdated changes to the values in PDS without citation or justification. If that's true, then somebody needs to be fired. Why? Because it creates the opportunity for some poor Ph.D to have her thesis review rejected because "This isn't what that reference says at all!". That's not an ethical gray line, it's wrong.
What makes this painful is that he's on his third round of court defended FOIA requests trying to get the data required to reproduce this publicly funded science. It's a problem.
(Score: 2, Informative) by Anonymous Coward on Monday June 25 2018, @02:55PM
You could be right. But data analysis pipelines can be messy with customized software installs, intermediate data that has suffered bit rot, etc. Ideally, you could just download a source code archive and run the analyze script to see what happens, but most scientists are not computer geeks, and version control is something they are still learning about. So I would tend to cut them some slack. Hitting them with FOIA requests seems like a tool from a political playbook rather a scientific one. If the above poster is right, Myhrvold can post his own database on a public server, and if other scientists find his analysis convincing, they will use his numbers. He will be vindicated and NASA will have egg on its face.
The claimed errors of 35 % do not seem particularly bad for this kind of data, and the abstract of Myhrvold's 2018 Icarus paper reads like the complaint of a nitpicker from hell. I'll note that when I read Mhyrvold's Medium.com blog post, every link to a supposedly different scientific paper linked to the same Icarus one.
(Score: 1, Informative) by Anonymous Coward on Monday June 25 2018, @03:42PM
I'm reminded of some anti-evolution guy who was submitting numerous requests to a cell researcher who had demonstrated some aspect of evolution in cell lines. The guy had no facilities to accept cells, nor any record of scientific credibility but argued it's public funded research so they have to give it to me. Not to mention, obviously, that complying with all information requests is effectively a DDOS on the research.
Sorry I forgot the name of the people involved. At the end of the day, the lead researcher just laid it out there and said we will share our data and cell lines with anyone as long as they have the facilities to handle it. I presume the anti-evolution guy didn't get the facilities nor go away quietly, I'll go with suing in court somewhere, In the name of scientific integrity, of course.