Why data don’t have all the answers with data scientist Andrea Jones-Rooy

Listen along

ReThinking with Adam Grant
Why data don’t have all the answers with data scientist Andrea Jones-Rooy
December 6, 2022

[00:00:00] Adam Grant:
Hey everyone, it's Adam Grant. Welcome back to ReThinking, my podcast on the science of what makes us tick. I'm an organizational psychologist and I'm taking you inside the minds of fascinating people to explore new thoughts and new ways of thinking.

My guest today is uniquely skilled at making data fun. Andrea Jones-Rooy teaches the flagship course Data Science for Everyone at NYU's Center for Data Science. Andrea also happens to be a circus artist and a comedian. It was the audience's runaway favorite speaker when I co-hosted the virtual Wharton Future of Work Conference earlier this year.

Here's a little taste of what made Andrea such a hit.

[00:00:44] Andrea Jones-Rooy:
I'm a time traveler. That's right. You heard this right? I would never lie to you about such a thing. I am a time traveler. It's very dangerous for me to be here right now. You know what happens when the space time continuum is disrupted, but I have come back from the year 2322 to give you several very important messages. So listen carefully. Now, I know that back in 2022, there's a lot going on in the world and we're really nervous about what the future's gonna look like, so I am here to share some good news. Number one, yes, Law and Order SVU is still on the air. We're all very excited for season 224. The future looks good. Oh, oh, oh. I should have also told you, by the way, the world does still exist. So well done all. But let's go back to this data science piece because that's gonna make the difference. Everyone gets involved in data science, and this leads to all kinds of wonderful things. I want to remind you, data is not magic. It is but a partial, imperfect snapshot of the past that we hope will tell us about the future. By the way, one more secret. Kim Kardashian did get a real job, and I'll see you all at work.

[00:01:54] Adam Grant:
So Andrea, I would love to know how you got into being a data scientist.

[00:02:00] Andrea Jones-Rooy:
I got into being a data scientist-- the answer I give my students, and that always makes them upset, is a series of poorly thought out decisions. I didn't do it on purpose. I graduated from undergrad with a degree in International Relations and Econ and Chinese, and I wanted to go to grad school cause I didn't know what else to do. I was one of those, by some miracle, got into the University of Michigan where we, I think barely overlapped.

[00:02:24] Adam Grant:
Go blue.

[00:02:25] Andrea Jones-Rooy:
Yeah, go blue. And there, I thought I was going to write sweeping essays on the history of war and the nature of human evil versus good and all that stuff. And I got there and they were like, oh, actually it's mostly statistics and game theory. And I hadn't taken a math class since high school and I was very proud that I placed out of any math in college.And I was like, I'm never looking at an equation again. Got to grad school and they were not joking around. So I, I, for years resisted it, but through a mix of stubbornness, lack of ego to leave graduate school, I eventually came to realize that I could understand math to some extent, and I was interested in turning the world into numbers and then doing interesting things with those numbers.And so I did finish with a PhD degree in political science with a focus on complex systems and taught as a political science professor for some time. Left academia. I was on a tenure track path. I left academia. I thought, I wanna see what the real world is all about. Do we need political science out there in the world? And it turns out we do need political science out there in the world. But no one wants to hear about political science because they think I'm gonna argue with them about conspiracy theories and who killed JFK and all that. And as I was working with companies, I realized that what I was doing was actually data science.

And as I was coming to that realization, NYU called me and said, "Hey, we're launching an undergrad program in data science. Would you teach our opening course called Data Science for Everyone?" And I said, oh my gosh, yes. And so I've been a data scientist ever since. I, I do data science very much with a social science angle, so I'm particularly interested in how do we thoughtfully turn complex and messy, abstract things, privilege and justice, into data in a way that is representative of the world, but still numbers that we can do cool things with.

[00:04:10] Adam Grant:
For a data scientist, you have, I would say, a slightly unusual stance on data.

[00:04:15] Andrea Jones-Rooy:
I do.

[00:04:16] Adam Grant:
Um, namely, you don't love them as much as most of the data scientists that I know.

[00:04:21] Andrea Jones-Rooy:
Right? The thing that gets me revved up around data is I was working with all these companies. Big, global, fortune 500 tech, whatever, fancy people, and they would all come to me and they'd say, well, what does the data say? The data tells us a story. And I was just like, I feel like that isn't how it works. And I really, it rubbed me the wrong way. And I thought about it for a long time, eventually wrote an essay about it. And this is something that I absolutely credit the University of Michigan political science program, cuz we spent all this time thinking very hard. Like how do you turn something into how I, my viewpoints on politics, my sacred values, how do you turn that stuff into numbers? The answer is it's not obvious. And there are a million different ways to do it. And my decision over how I'm gonna turn something like my political identity or my satisfaction at this company, or how well someone is performing at this company into a number is going to be the result of a whole bunch of biases in the person collecting the data, in how we've decided to measure it, in what we're looking at when we think about performance or top talent, or how good our government is at doing what we think it's supposed to be doing. And all of these biases go into the data itself. And so I really. Came to this point where I was like, don't say what does the data say to me anymore, right? We say things about data, we tell stories using data. We hope that we're doing it in a way that is thoughtful and transparent and accurate, and reflective of the world, but it's never gonna be perfect. And I really feel like the excitement about data science is awesome. But I've started to get quite nervous and as you can see, worked up about the fact that a lot of times we take data to mean truth, capital T, truth, when really it's a description of the world that someone has bothered to write down.

[00:06:02] Adam Grant:
Yeah. I remember reading that essay of yours. It was, it was actually how I discovered your work. And I came away with this, this very clear sense that the answer to what did the data say is nothing.

[00:06:13] Andrea Jones-Rooy:
Yeah.

[00:06:13] Adam Grant:
Data don't talk. They can't answer your questions, they can only inform your questions. Right. And it's you as a human who collects and interprets the data.

[00:06:21] Andrea Jones-Rooy:
Just like maps don't tell us where to go. We look at a map and from there deduce a route that we wanna take or, or we trust an algorithm to deduce that route based on something that humans have asked the algorithm to do with the information in that map. And I will say that there's a lot of really important conversations going on around algorithmic bias, machine learning bias and all of that. And that's super important. I basically was like, yeah, but there's another problem. The data itself is flawed, biased, systematically wrong in some ways. But the, the challenge, and you know, I said this in that essay and, and I try to emphasize it, is, that doesn't mean we throw data away, and it doesn't mean we reject data that we don't agree with.

[00:07:02] Andrea Jones-Rooy:
I sometimes try to post little snippets of interesting data sets on the internet, which is a huge mistake often because people will come on and say, nah, that data's biased. And you're like, well, okay. Yes. All data is biased, but we need to talk about biased in what way? In what direction is that gonna push the results? Is that gonna overcast some other thing that we're missing? Like you can't just throw data away because you don't like it. You have to systematically say, oh, I think we're probably overrepresent this group. We're probably underrepresenting this. We're probably actually measuring this, which is close to what we're trying to do, but not exactly. It's not enough. I do worry in misinformation land to be like, eh, that data's wrong. Like, ugh. It's not that simlpe. Science isn't perfect. I think one thing that makes me nervous, I'm gonna alienate just people on all sides of the political spectrum. One thing that makes me nervous is, is friends of mine who are very, let's say pro-science, which is kind of a term that makes me nervous already, but what we'll see, things like, believe scientists, trust science. I don't think you need to do that. That would make scientists lives easier. But I think you can read science, see the decisions that were made and say, oh, I see why they did it this way. I wonder what would happen if they did it this other way. And then either you can go do it, or more likely you can find a study that did it or trace your way to a meta study that makes sense of the whole thing.

[00:08:14] Adam Grant:
Yeah, I like that. I would like it more though if we could do this with a version of the Rawlsian veil of ignorance and apply this to evaluating research. So for Rawls, if we want to know if a society is fair, we ask people, would you join this society even if you don't know your position in it? Well, I think we should do the same thing with science, which is to say, do you think this is a quality study based solely on the methods? Before you know the results. Do you think this is a rigorous way of answering the question? And if it is, then you have to accept the results, whatever they are. And that's how I try to evaluate science generally, right? I wanna read the method section before the results section because I know that if I have an opinion about the topic or I have any emotion around the topic, then I'm gonna be tougher on the studies that contradict my beliefs and fall into a confirmation bias zone very quickly. And so why don't we separate the methods from the results and put the methods out there and then say, "Hey, if you think this is a rigorous study, then you have to believe whatever it shows."

[00:09:12] Andrea Jones-Rooy:
One thing that always rubbed me the wrong way, even in grad school was, was, I was like, academics don't have any sense of suspense. You get the whole punchline in the abstract. And that's convenient for grad students and professors who need to sift through a whole bunch of papers really fast. But I think you're onto something, both from a scientific integrity perspective and also from a, maybe more of us would read science if it was like, how's this gonna turn out? Right? Like, what's the what? There's a twist. There is no effect. Ah, I think there's something really powerful there.

[00:09:40] Adam Grant:
I like that. Yeah. I love the idea of putting mystery back in science.

[00:09:44] Andrea Jones-Rooy:
Yeah.

[00:09:45] Adam Grant:
So I wanna, I wanna come back to the observation that you made about trust science, trust scientists. I want people to trust the scientific method.

[00:09:53] Andrea Jones-Rooy:
Yes.

[00:09:53] Adam Grant:
As the most rigorous tool humans have ever created to accumulate knowledge.

[00:09:59] Andrea Jones-Rooy:
Yes.

[00:09:59] Adam Grant:
Right. And I think of science as a systematic process for ruling out false ideas and moving closer to the truth. And if people don't trust that, I don't know what we can trust.

[00:10:08] Andrea Jones-Rooy:
Right, right.

[00:10:09] Adam Grant:
Like think, think about evidence-based medicine and how far we've come in the last century, literally doubling the human lifespan because we stop just saying like, let's do you know, I have an idea. How about a frontal lobotomy?

[00:10:21] Andrea Jones-Rooy:
Right.

[00:10:21] Adam Grant:
Um, I think maybe, maybe you have bad blood. Let's just, let's just let some of that out and hope that you get cured. Well, no, we're gonna use the scientific method to try to figure out what's wrong and what might be, right.

[00:10:33] Andrea Jones-Rooy:
Right.

[00:10:34] Adam Grant:
And. I, I, I guess I've started noticed some telltale signs that people don't understand the scientific method. One of them is when-- this is so common on social media. When I see the knee jerk reaction, correlation isn't causation. Yeah. Whenever someone says that it's a red flag for me that they don't understand, at least how social science works, because number one, as a social scientist, if I made a causal statement, I could promise you that there's good reason for it. Either it's a randomized controlled experiment that does prove causality or at least support causal inferences, or in some cases it's impossible for any other relationship to exist. Like if there's a correlation, you either need X to cause Y, Y to cause X, or there's some common cause of Y and X, right?

[00:11:21] Andrea Jones-Rooy:
Yeah.

[00:11:21] Adam Grant:
And if, if the latter two have been ruled out, we can start to assume that causality is present. And I don't need to tell you this of course, but I find myself wanting to rant this way and I wonder if you have, as a data scientist, better ways to educate people around this. "Okay. Wait. Sometimes a correlation does mean causation..."

[00:11:40] Andrea Jones-Rooy:
Mm-hmm.

[00:11:41] Adam Grant:
"...and you can't just debunk a study or a body of evidence just because you don't think that the correlation means anything."

[00:11:48] Andrea Jones-Rooy:
The scientific method has been, I don't, I don't know if it's been, it's been misunderstood for a very long time. I grew up with the wrong understanding of the scientific method. My first introduction to the scientific method was in third grade, and we had to dissect owl pellets, which are disgusting things that owls throw up. And we had to do it using the scientific method and say, hypothesis, this, this, this. And I thought, this is so boring. I just, cut these gross things apart and see what's in there. And it wasn't until, again, that I was in grad school that I was like, oh, if we have a whole bunch of evidence in front of us, we don't know where to look. And I say this to my consulting clients with data science all the time where they say, what are the insights? What are the discoveries? It's like, well, you have to have a question about the world. And then you can look for evidence to see if it's consistent with what you think is going on. And sometimes, as you said, correlation is the best that you have, and for many reasons, it's not plausible or ethical or anything to do a randomized controlled experiment where say you randomly assign a whole bunch of teenagers to smoke cigarettes for 10 years and see what happens, right? So we have to look at correlation, and you're absolutely right that we should be skeptical of correlation, but it doesn't mean that correlation is not causation also doesn't mean it's not, not causation. I mean that person on the internet was right. Right? And the scientific method too is, I'm so glad you brought up cuz it's not magic. And I think sometimes people think about it as like this magical, mysterious thing that you can only do in a lab coat. Literally all it is is I have a question about the world. If I'm right, I should see this. Do I see this? Huh? I do. How else can I explore that question? And that's kind of it.

[00:13:22] Adam Grant:
Agreed. So going back to this point you made about science, kind of like delivering an aha.

[00:13:30] Andrea Jones-Rooy:
Yeah.

[00:13:30] Adam Grant:
As opposed to just having a headline. I wonder if you could connect this to your, your other career as a comedian.

[00:13:36] Andrea Jones-Rooy:
Oh boy. [Laughter]

[00:13:37] Adam Grant:
It seems like a version of this, this problem that we're, we're discussing is that science basically leaves with the punchline.

[00:13:45] Andrea Jones-Rooy:
Mm.

[00:13:46] Adam Grant:
And there's no setup.

[00:13:48] Andrea Jones-Rooy:
Right.

[00:13:48] Adam Grant:
And I wonder if you could talk about that.

[00:13:50] Andrea Jones-Rooy:
That's brilliant. Yes. Comedy hinges on surprise and an unexpected twist. And so you set up saying something and everyone says, I agree with that. And then you say something that nobody expects, and then it's very funny. Hopefully it can be not funny at all. And then it's very awkward, right? We do science like we do standup comedy where we say I think this might be an interesting thing. I mean, standup comedy is hypothesis testing in real time, right? I will sit ahead of time and say, oh, I'm gonna say I'm gonna talk about this. And my hypothesis is that if I say that, people are gonna laugh. If they don't laugh, I might say, okay, what if I set it up the same way but say something slightly different? Oh, people are laughing a little bit more. If no one ever laughs you say, ah, maybe the premise is all wrong and you're testing all the time and you're getting the feedback in real time. And scientists are kind of doing that and the part of the fun of doing science is that it is surprising and the feeling for me at least, is when I'm going through like a Jupiter notebook and I'm working in Python and I'm, I'm doing an analysis, say for a company and I'm ready to click, go on something to see what the predicted effect of whatever, like to see the result, it's a suspenseful, exciting moment, and it's that exact same tension that you feel in a comedy room when you're really killing it on stage, where you've got the audience and they're held and we're all held in suspense. You're like, what's it gonna be? And you're like, negative correlation, punchline and ah! Right? One of the reasons I'm so interested in standup comedy and love watching standup comedy is that the precision in standup comedy is absolutely right up there with the precision in data science or any scientific analysis. And if anything, it's way more sensitive and you know, you do an analysis a bunch of times and you keep getting a correlation every time unless you really change the variables. Stand up comedy, you know, you pause for too long and you lose the crowd and it's over.

[00:15:38] Adam Grant:
Andrea, are you ready for a lightning round?

[00:15:39] Andrea Jones-Rooy:
So, ready.

[00:15:40] Adam Grant:
Okay. What is the one book we should read if we wanna learn to think more like data scientists?

[00:15:45] Andrea Jones-Rooy:
This'll cause a stir, uh uh, The Structure of Scientific Revolutions.

[00:15:51] Adam Grant:
You're gonna recommend Thomas Kuhn.

[00:15:52] Andrea Jones-Rooy:
I am gonna recommend Thomas Kuhn. I understand--

[00:15:54] Adam Grant:
--even though he got so much so wrong?

[00:15:56] Andrea Jones-Rooy:
Yes. I did not understand that science is fun until I read that book. You don't, you don't have to read the whole book, but skim it. And I thought he did get a lot wrong, but I think that the, the humility that comes with totally changing our worldview, even though we think it's very, very set in stone, was something that was a revelatory to me.

[00:16:18] Adam Grant:
That's actually a really compelling rationale. I remember reading it is as a freshman in college, I guess in some ways it planted the seeds for Think Again because I was horrified that entire communities of scientists, they stuck to their, their guns instead of believing the evidence. Yeah, exactly. I remember it really bothering me. I never made this connection until you brought it up, but it really bothered me that like, that the, one of the, the main arguments to that book was that science progresses one funeral at a time.

[00:16:44] Andrea Jones-Rooy:
Yes.

[00:16:45] Adam Grant:
I'm like, well, why does it have to be that way? A good scientist would change their mind when presented with new data.

[00:16:49] Andrea Jones-Rooy:
Exactly. Exactly. Exactly.

[00:16:51] Adam Grant:
Maybe I should study that. Okay.

[00:16:53] Andrea Jones-Rooy:
Yeah!

[00:16:54] Adam Grant:
What's your best life lesson from being a circus performer?

[00:16:58] Andrea Jones-Rooy:
Best lesson I learned from someone named Bendi Bandini, who is a contortionist and sword swallower. He swallows two swords. Have you heard of that? Two swords. He was teaching me how to do a bed of nails. And the bed of nails is exactly what it sounds like. It's, um, a whole bunch of wood with a whole bunch of nails sticking out of it. You know, kind of a square. Very big, very serious nails. And I was like, surely there's some trick. You're not really lying on the nails. You this, you that. And there is some, you know, there's some basic physics to it where you wanna be straight down and not rub back and forth, you know, to not rip your skin off. But it still hurts no matter what. His advice to me, he was like, I was about to go on stage and do it for the first time. And he was like, here you go. All you need to know is it's just pain. And I was like, that's not very helpful, is it? Like it's gonna hurt. That's not great. But it actually really helps because in the moment, as long as that's not to excuse serious amounts of pain, but in the moment it's like, it hurts, but it's okay. And it's not dangerous, it's just pain. And so I actually at the time, dismissed it as just barely helpful. But since then, this was about 10 years ago, I think about it all of the time when I'm upset about something or frustrated about something, or scared or nervous, I think, well, it's just pain. And that actually helps me get through it. Unless it's life threatening and usually it's not. Luckily.

[00:18:16] Adam Grant:
Speaking of pain, agree or disagree?

[00:18:18] Andrea Jones-Rooy:
All right.

[00:18:19] Adam Grant:
This is my thesis. Voting polls are a giant waste of time.

[00:18:25] Andrea Jones-Rooy:
Disagree. Because we can build from them and improve them. There's hope for a better version. So I don't think they're a waste of time yet. I think we should spend less time, but I wouldn't, you know, I interpret a giant waste of time as like, let's kill the whole enterprise, and I'm not there yet.

[00:18:40] Adam Grant:
Oh, I am definitely there.

[00:18:41] Andrea Jones-Rooy:
Okay, really? If nothing else, it's a good way for me to talk about selection bias with my students.

[00:18:46] Adam Grant:
There you go. Yeah, I respect that.

[00:18:47] Andrea Jones-Rooy:
All right.

[00:18:48] Adam Grant:
All right. You're such an entertaining presenter. If I want to be more entertaining, if I wanna be funnier on stage, where should I begin?

[00:18:56] Andrea Jones-Rooy:
Call out what you think is awkward in any situation you're in and people will laugh. You know, you don't have to be giving a com, you know, a comedy, like a standup set or anything, but giving any kind of presentation and, uh, you know, the microphone's not working. Rather than say, oh, I'm freaking out. Just be like, well, the microphone doesn't like what I have to say. That's not that funny. But like the weirdness.

[00:19:21] Adam Grant:
What's something you actually want your students to write down?

[00:19:25] Andrea Jones-Rooy:
Science is a series of steps taken by people committed to being honest and transparent about those steps.

[00:19:37] Adam Grant:
Wow, that's so interesting. I'm just thinking about the committed to being honest and transparent. Yeah.

[00:19:45] Andrea Jones-Rooy:
Do you disagree?

[00:19:46] Adam Grant:
No, I don't disagree. I just, I always think about science much more in terms of the, the rigor of the methods to pursue the truth as opposed to the transparency with which we communicate what we did. I think it has to be both, doesn't it?

[00:20:02] Andrea Jones-Rooy:
It's definitely both. My friend Natalia Reagan is a science communicator and she talks a lot about science is a verb. Science is something that you do, and I really like that. And so I think. This, this saying, I did these things. You can replicate these things and I did it with a, you know, best of intentions to get to the truth. Even though I know, cuz I have humility that I never will and you're following the scientific method. I think it's science, regardless of whether you're using a hammer or the fanciest, I don't know anything about construction. The fanciest, you know, bulldozer you've ever seen. Science seems so mysterious to so many people and I wish that it weren't, and one way to make it not so mysterious is to do it. And I think rather than trying to simply, I think it's very important, to get the news out about cool studies. I think your idea of making it more of a mystery is a great one. You know, and sharing some of the interesting work that's going on is, is extremely important. But I would love, my vision is for everyone in the world to at least have some idea of how to take a piece of data that they're interested in and get as far as generating a correlation or a simple linear regression and to know how to interpret it. That's a huge ask. But I would love if we all just had that I think we could spare, at minimum ourselves, some annoying conversations on Instagram.

[00:21:26] Adam Grant:
[Laughter] It'd definitely spare me, and I'm seriously tempted to launch a campaign and I'm not a campaigner and I don't do this, but I'm, I believe in this so strongly that I would consider it, at least, I want to replace trigonometry with statistics.

[00:21:40] Andrea Jones-Rooy:
There we go. Now we're cooking. Yes.

[00:21:42] Adam Grant:
Like statistical illiteracy is a massive problem.

[00:21:46] Andrea Jones-Rooy:
Yes.

[00:21:46] Adam Grant:
In our time.

[00:21:47] Andrea Jones-Rooy:
Yes.

[00:21:47] Adam Grant:
And statistics are so much more useful than sines and cosines. Which, I had to think for a second to remember what those words were.

[00:21:54] Andrea Jones-Rooy:
Same.

[00:21:55] Adam Grant:
Right. Which tells you how recently I've used them. But even if I weren't a social scientist and you weren't one, we would use statistics all the time to interpret information. So what's it gonna take? How are we gonna make this part of people's education?

[00:22:09] Andrea Jones-Rooy:
First of all, I think your suggestion is spot on. Though let's both get ready for the hate mail from the trig community out there.

[00:22:16] Adam Grant:
[Laughter] yeah. Big trig is gonna kill this idea. They're gonna hate it.

[00:22:19] Andrea Jones-Rooy:
Yeah. We're getting canceled by big trig. I love it. I, I mean, I think this goes back to your opening question for me, which was how did I become a data scientist and it was a series of accidental decisions and I feel extremely lucky. Sometimes I wonder if the whole thing was a huge nightmare, but I, I, I generally on average, feel extremely lucky that I stumbled into this quantitative social science program. Literally only because I didn't read the program description. Okay. But I was forced to learn all this stuff, and it took like four years before I was sitting there and I was like, oh, this is awesome. And I'm not a math whiz. I did not grow up loving math. I, I probably wouldn't have done well had I been given the opportunity to take a statistics course in high school. But I feel so grateful to have been given these powers. I mean, this is an actual superpower, is to be able to look at data and make inferences about the world and be thoughtful about the strengths and weaknesses of those inferences and have ideas about where else. So like that's, that's a superpower at NYU where I teach, you know, a data science undergraduate program didn't exist five years ago, and now it exists.

So we're kind of baking it in, but. I'm still kind of attracting students who already think they love math and now, but I wanna get to the, I wanna get to the artists and the, the policy makers and the future NGO leaders. I wanna get to everybody and, and maybe, you know, you and I and anyone else who cares about statistic and there are a lot of people doing this, you know, the more that we can talk about and invite people to join in to the fun of statistics and get excited about doing.

The better off will be, as opposed to saying, you're dumb. You need to learn these things. Statistics is hard. The other thing I'm doing is a lot of companies are hiring data scientists to do their analyses. Great. Why not? I really want everybody else at the company to also, and as you know, I work with people, teams a lot to also be able to do even very, very simple analyses. People who, say someone who's been on a people team or HR for 20 years, they have amazing ideas about what might be working at their company and what might not be working. Of course, just their ideas alone, you know, everyone's ideas are valuable. If each one of them were also tasked with doing some analyses and actually working with data from a bottom up perspective, I think we would learn so much more than if we just leave it to a small group of data scientists who might be technically very skilled, but not have a lifetime of experience to say, well, actually the interesting question would be this, what we really would wanna have data on is that. And getting those two groups to work together, I think is another thing that we can do to kind of bridge this gap where right now it's too much of the haves have nots in statistics.

[00:25:07] Adam Grant:
I love the vision. I, I'm completely on board, you know, with the fundamental idea of saying, let's take people who don't think that they're good at math or who don't use statistics as part of their jobs and help them become more informed and actually see that this is a way of thinking. One of the things that I worry about when people start down that path, they sort of fall off the tight rope, and I say tightrope intentionally knowing that you're a circus performer.

[00:25:36] Andrea Jones-Rooy:
I've been on the tight rope this whole conversation. It's pretty impressive, right?

[00:25:39] Adam Grant:
Have you?

[00:25:40] Andrea Jones-Rooy:
[Laughter] Yeah. No.

[00:25:41] Adam Grant:
I feel like people fall off the tight rope often between curiosity and skepticism, and they don't understand that you need both to be a good data scientist, right? So I think obviously you need skepticism to apply your critical thinking capabilities and say, well, how might this conclusion be incomplete or maybe misleading? What other information is missing? But then, not to stop there, but also to say, do the strengths of this, this study outweigh the limitations. Did it teach me something I didn't know before? Um, and is there value in one of the lessons I can glean, even if I don't agree with every single choice the researchers made? And I feel like I see a growing number of people falling off the first side. They think they're being critical thinkers, but they're actually just being cynics and saying, I don't believe this community of experts and experts are not trustworthy. And I think for myself, I do my own research. Yeah, exactly. Yes. Keep doing that and I hope you, you also go and put your, your own research through a double blind peer review process and have your thinking skewered too. It's fun.

[00:26:46] Andrea Jones-Rooy:
I think that, you know, we're, we're doing our own research and we're looking it up ourselves and we're being skeptical and therefore using that to reject any result we don't agree with-- that's, you're right. That's not science. And, and the way that I, I tend to try to talk about it because the skeptical community is its own, you know, broad community and there's a lot of really great work there. But I think you're absolutely right that people use it as an excuse to reject whatever they don't agree with is humility and fearlessness.

So don't be afraid to dig into a study. Don't be afraid to contact a scientist and ask how they did it or, or, you know, maybe they're gonna get mad about this, but, but actually say, I disagree with this outcome. What is it about their methods that I think is so problematic? Do I think they missed this group? Do I think they mismeasured this? Do I think-- don't, they're probably not just lying to me, right? Like skeptic, like cynicism is eh, it's all crap and the whole thing is garbage. Like that's not how we move forward as a society. We move forward as a society by reading a study and really thinking, what would I do differently?

How would I improve this? And I think as long as we don't equip people with those skills, we shouldn't be surprised that they're falling on the tightrope side of critical thinking, equal skepticism, because we haven't given people this scaffolding to evaluate how science works. I wouldn't know how science works had I not stumbled into this program, and I don't know I mean, okay, maybe in addition to a statistics class, we also need a research design class in, in high school or something like that because we're we're--

[00:28:20] Adam Grant:
--yes.

[00:28:20] Andrea Jones-Rooy:
You know, we're, we're saying, "Hey, look at this house", and people are like, "I don't believe in houses." And you're like, whoa! Right? I think as long as, yeah, we don't give people those tools, we're either gonna end on the side of belief or reject any science you don't like. So the fearlessness and humility required to do science is something I wish we all shared more. Again, not just our results.

[00:28:44] Adam Grant:
I am, I am aligned to the point that I'm ready to, to cheer, all right? But I'm not gonna do it. I think the most important thing that I can say in closing is, I mean, this is, I'm a little hesitant to go out on this limb but I'm just gonna do it.

[00:29:00] Andrea Jones-Rooy:
Okay.

[00:29:00] Adam Grant:
Big trig, we're coming for you.

[00:29:02] Andrea Jones-Rooy:
Yes! I will stand by that.

[00:29:06] Adam Grant:
Thanks, Andrea.

[00:29:07] Andrea Jones-Rooy:
Thanks, Adam.

[00:29:12] Adam Grant:
I'm excited about this idea of bringing more mystery into the presentation of science and I think so often we just, we lead with the punchline . We start with the results. We would be much better off in many cases, showing the sophistication of the methods we use and actually showing our work, just like you would do in a math problem, and then allowing people to understand, "Hey, you know what? We put a lot of thought into how we measure this, and that makes the conclusions we generate a little bit more trustworthy."

[00:29:46] Adam Grant:
ReThinking is hosted by me, Adam Grant, and produced by TED with Cosmic Standard. Our team includes Colin Helms, Eliza Smith, Jacob Winik, Michelle Quint, Sammy Case, Banban Cheng, and Anna Phelan. This episode was produced and mixed by Cosmic Standard. Our fact checker is Hana Matsuidara. Original music by Hansdale Hsu and Allison Leyton-Brown.

[00:30:08] Andrea Jones-Rooy:
I have a personal vendetta against pie graphs for no reason other than I think I've never seen a pie graph that was an improvement over a bar chart, but I'll, I'll let your, uh, audience come at me for that strong claim. Even How much pie is left? I think you could put in a bar chart.