Iulia Georgescu

AI has great potential because it's transdisciplinary. But we are using it perhaps

in a bit too narrow way without seeing the bigger picture. And we're also too much focused on utilitarian and short-term goals rather than trying to see, this is a great tool that we have, integrative tool by design, know, what are we going to do next? What is the bigger picture? How can we use it into more integrative constructiv...

constructive approach that actually brings disciplines together rather than pushing them apart

Welcome to Taste Bench. I'm Philipp Zahn, my co host is Bruno Marnette and this is a podcast about taste and the creative side of AI. Today's guest is Iulia Georgescu, a physicist who spent more than a decade as an editor at the Nature Journals. Her current work draws on the history and philosophy of science to understand scientific judgment, tacit knowledge, and how AI influences the production of scientific knowledge.

We are very happy, Iulia for you joining. Thank you for taking the time. we first want to start with what about you? What are you doing, and also your experience as an editor? You have been a long time with Nature and worked in several capacities. So maybe can you briefly describe your journey from what you have been doing in the past to what you're doing now?

Thank you. Thank you for the invitation. So I'm a physicist by training. I've done I had the usual academic trajectory where you go to PhD and then you do a couple of postdocs at which stage I wanted to see like the broader picture. And for that, like I joined Nature Physics as an associate editor. And I stayed with the Nature journals for over a decade.

as an editor for Nature Physics, selecting the papers that were published in the journal and then as a launch editor of Nature Reviews Physics where I created the publication, launched it and drove it for a few years. So these days I'm very much interested in doing research myself in a personal capacity as a side to my day job.

And I'm very interested in using the history and philosophy of science approaches to understand science as an ecosystem, but also AI in particular and the changes it brings. And I have a particular interest in the history of computation and the history of computational physics.

We talked to a lot of scientists. and I'm not sure how to exactly put it, but scientists have to publish, and they're very often not that satisfied with the publication system and the peer review system behind it. Can you maybe describe a bit your own perspective on how you when you were working in this capacity, how you did your

how you defined your role and how what was what was your view on what's your job and how it should be done.

Yeah. So

there's a very good question that allows me to go into also into historical anecdote. First, like when I started as an editor in Nature Physics at my very first conference, people gave me like a hard time asking like, how do you select papers? What gives you the right to choose papers? Like, how do you think these are good? And how do you think, who are you to judge us basically?

And I tried very hard because I was a young editor, very passionate about what I was doing to convince them that we had a method and it was right and was fair and it was good. And actually it's a lot of subjectivity because there's lots of uncertainty.

But as an editor, I would publish a paper and then a colleague would come to me and say, you published this guy talking about this topic. That's such good taste. And I would be very smug about it. So what is actually editorial taste? after, when I was no longer an editor and I was doing, as I said, historical research, I came upon this example that is really interesting. And I wrote about it in Physics World. So in the 60s,

scientists were unhappy with how a peer review was taking place. It's nothing new. It was too long. Didn't give the results they wanted. So two scientists at Bell Labs, Phil Anderson, who would later get a Nobel Prize in Physics, and Brent Matthias decided they've had enough of this. So they would make their own journal. They would select what they thought was interesting and they would publish what they thought was worth it and even pay the authors.

And people, when they heard about their idea, they wrote back and said exactly what many years later people told me as a junior editor, who gives you the right to judge us? Why do you think it's a good selection? How are you going to do this in a fair way? And in the first editorial of the journal that Anderson and Matthias produced, they said like, this is not the selection. This is

a selection of what we think is interesting, but it's not necessarily the right thing. And that's basically taste. And they had really good taste. They published some really good papers, some very influential papers in physics. The journal is completely forgotten because it was only published between 1964 and 1968. It was called, very interestingly, physics, physics, physique.

Physika, I can't pronounce Russian correctly, but like in three languages, the same thing in three languages. And this was very short lived publication because actually the realities of the infrastructure needed to produce journals and to actually constantly assess and peer review and published articles, it's actually much more complicated than people think. So it didn't live very long, but it did publish some really interesting things.

And one example is the famous Bell theorem, which was quite influential in quantum mechanics. it's the fact that the Bell test existed helped the development of quantum technologies. And the story is very interesting. So one day they get this paper by a guy no one has heard of, John Bell. And Mathias doesn't have a clue about.

this paper Anderson says, like, I will publish it because it would probably spell the end of Bohmian mechanics, which he found distasteful. So he published maybe for a dubious reason, but the choice was actually very good because this paper was later found by John Clauser who came to this journal because it had a weird name. Discovered Bell's paper, a few years later did the Bell tests.

which got him an Nobel Prize in Physics in 2022. So I think that's an interesting story, an interesting story that tells you like, you know, being an editor and assessing science is, you know, it's not, it's part, you know, you have to be consistent, but it's a bit of an art. It's hard science in a sense. And there's a lot of your personality, your intuition, your taste.

in doing that. And it's subjective, it's flawed, and perhaps it's not fair, but that it's what taste is, ultimately. And this is why we like tasteful selections. Some people might not enjoy them as we do like in art, in literature, and so on. But that's why I think it's a ill-defined metric, but an interesting metric that gives more than

well-defined metrics like, you know, let's optimize for things that will be well cited or things that will be downloaded a lot or things that will be picked up a lot in main like media or like, you know, papers that are long or papers that are on this subject or papers that are by famous people. You know, you can have all sorts of criteria you optimize for. But once you set up on a specific criteria, you narrow yourself down. Whereas,

ill-defined measure actually allows you to explore a much broader space. And again, sometimes you make brilliant decisions, maybe not for the best reasons, but that's what makes editorial taste in this case interesting.

when I'm informed correctly, you actually founded a new journal, right? While you were working there.

I did found a new

journal which lasted longer that Anderson's. It's still going on, which

Congratulations.

is a good thing. of course, it's a very different thing. again, Anderson and Matthias they were just doing it two people by the side of their job as researchers at Bell Labs. were a full team back by a big publishing house. It's very different way of launching a journal.

Why did you do this? So what was the motivation behind it?

So the motivation was not necessarily a personal motivation, like as you know, like there are more more more journals every year to serve like different communities. And that's actually again, like going back to historical thing, like why do we have disciplines? I never asked myself this question before, but it's actually very interesting because there weren't disciplines until the...

19th century was just natural sciences. And then as the amount of knowledge increased, people had to break it down into manageable chunks. And as they did that, the disciplines emerged and in universities where people were teaching this knowledge, they had to do the same. then so departments and divisions and other administrative

categorizations appeared and they were established and around them were established communities. Journals started to specialize to serve those communities and then science became like natural sciences, they became like physics, chemistry, biology and so on. As the knowledge increased even more, like we had to have like smaller disciplines and smaller disciplines. And again, the journals would reflect like again, these sub-disciplines

But the knowledge kept increasing. we have now tens of thousands of journals that publish, I think, over six million papers a year,

On the one hand, you need specializations for all these subfields. But on the other hand, you also need something cross-cutting. In the journal I worked on,

I launched was across these fields and trying to publish reviews that would show progress on bigger areas, but also interdisciplinary topics. So again, yes, please.

Is it difficult to start a publication in a world where the best scientists want to be in the biggest publications? Do you have to allow them to cross-publish or how hard is it to get?

I think it's increasingly

hard because everyone is very busy. All the journals want their famous people to publish in their pages, obviously. And again, like people have that much time. So what you need to do is actually to find the right people for the right topic, because if you pitch to them like a topic that is close to their heart and they think it's timely and

Mm-hmm.

they think it's interesting, they would some...

carve the time to do it. Or if you just go without having done your homework, without knowing what this person is interested in, what are the topics, what is interesting and timely right now, then that is very difficult.

So it sounds like you seem to bring your own taste, basically, when you approach people.

You do,

you do, because on the one hand, everybody knows this particular topic is timely now. But there's a flair. So like a colleague of mine was saying, a good review is like the right author, the right topic at the right time. But actually knowing that takes deep knowledge of the community, but also like a bit of a flair, like when is the timely?

Sometimes it's too early and people don't pay attention and sometimes too late and there are three other reviews on the topic. That's taste or flair or call it intuition. It's part of knowing the field and the topics.

And maybe just a quick follow up on this. you, I see how like a small publication would have all the right incentives to actually be tasteful and share their tastes and so on. Whereas maybe the bigger publication might need to become more bureaucratic and so on. Can you tell us a little bit about the, I guess the large publication? What do you think is the current set of incentives for them to either have tastes or try to be more of a box ticking kind of exercise?

Yes, so actually with the open access, is, know, like creating theory, the business model is shifted from subscription where like you basically pay for the publication, but what you get is in a sense like curated quality to big volume. So both from the publisher side and in a sense also for the people who are paying for the open access, you want volume.

But there's always a trade-off between quality and quantity. And actually, also in terms of how much work you can put into a big journal, like some way has to give. So journals like Nature Physics, which was funded in 2005, they were meant to be very selective. And again, like in the...

as like a tasteful selection of what is interesting in physics. You publish very few articles on particular topics that the editors thought were interesting. Unfortunately, this model doesn't work in the current day. Not only because of the open access pressure to publish more, but also

Mm.

because people don't read journals anymore.

and

People hardly read the articles these days because there's so many. And the idea of getting every month a publication that you browse and like, this is actually interesting. It's not in my field, but it's really exciting. I want to learn about that. I think there are very few, maybe a few retired professors who still do that, but

Mm-hmm.

that's it. So actually the model of the journal that someone reads the issues of.

is completely outdated. And unfortunately, it's unfortunate because I think there's value in that, in actually having a sample

Mm.

selection of what's going on in physics this month. I do enjoy reading the issues of Science and Nature and learning about different things, but I appreciate there are very few people who are even aware that issues still exist, that these are different publications.

Mm-hmm.

because the way we access this information is very different today, right? Through social media, through feeds, through like AI curated lists of what AI thinks are our interests and so on. that's these journals, like products of a different age that served like the information needs of different, of a different world.

I have a follow-up question, this what you just said. So in music, the concept of an album is obviously not the same anymore as it used to be. And what you describe with this kind of outlets, because I I still remember these times when you get, you know, the paper time when you get actually a journal. And I I agree, it was interesting to see because the taste of an editor, or in general, this process, maybe this was kind of accidental, but you you

You got this feeling there is something behind it, which could be totally wrong because it was just accidental and you know, the sequencing of it was according to other criteria. But yeah.

No, actually, sir, actually as someone

who's produced a journal, it's not accidental, it's deliberate, because every issue we had a theme. And the selection of articles in that particular issue was not accidental, it was by design, you know, and we'd have issues that explore certain topics, then we'd commission shorter articles to give context. But again, the sad thing is, like you put all this effort into curating this, and I think the analogy with an album is...

is to some extent accurate to make this product that is one thing. Then people just take one paper out of it and it's like, why did you publish this paper? Like, I don't particularly like it. It's well, yes, but like this is part I used to give this presentation is and I was showing like a piece of a puzzle and then I was showing the big puzzle that was the cover of the journal.

You have to see it in context. What story does it say? And if you're a long serving editor of that journal, what you do is you'd see like a field develop. And in the beginning, you'd think like I published these results that five years later, they weren't amazing, but there was the start. could already see like and I know like at least it's true for all journals.

You know, like you see a field start and you see it grow because people tend to publish where they've published before or where relevant papers have been published before. And then you see like a field expanding and growing in your journal. And, know, I know editors from the old times would say like, I published the first paper. I saw it grow over years. And, know, that's again a taste. You know, someone saw a new field and thought like, this is interesting. I want to give them

an outlet And again, like when you're talking about, you know, thousands and thousands of papers in a journal a year, you can no longer do that. So this only works if it's a very small journal with very few articles and are curated to a large extent by the editorial team.

Yeah, I think it's similar to monographs. And also the role they describe is almost a bit like a curator in an art show where you're putting things together.

It is. That's exactly

what it is. Editors are seen as gatekeepers. But I

Mm-hmm.

always wanted to say, we were not gatekeepers. Having your paper rejected is harsh. And I didn't enjoy it as a scientist and don't think anyone enjoys. The role is to curate it. And the curation is, as you say, it's a selection. It's not the selection because there's no such thing as this selection.

And this human curated format, know, some people might not like it and some people can argue against it, but again, it's a matter of taste. And like you have taste in music and like you have taste in art, you can have taste in scientific publications. And I have my favorite journals now that I like to look at. And I can't look at one of these.

exist mega journals because you can't make sense, it's just too much.

Yeah. we have not talked about AI in the publication process, but actually I would flag this and come back to this. because before that I want to touch on if it's fine with you on the production process itself, not so much on the outlets. maybe as a transition to this, you are also a researcher. and I think in my experience, you know, as a researcher

At the moment when I became a re a reviewer, my whole perspective on writing papers myself totally changed. you have a lot of experience as an editor. How did this actually influence how you think about writing papers or doing research in general?

I think it's a very, very important thing. We think of automating these days, but actually writing your first paper is a rite of passage. Reviewing your first paper is a rite of passage. Beeing on an editorial board is a rite of passage. And you can say, yeah, that's like old school. But this is how generations of scientists have trained. writing your paper, need to...

put your ideas in order in a coherent way and explain them to people who don't know about everything about your research in a compelling way. And then you think you've done a good job because you know a lot about the background and you have a lot of tacit knowledge. But then when someone else reads it, it's like, I'm not sure what this is obvious. So I don't understand all the steps. That's where the good editor comes in, because a good editor will challenge you. Hey, hang on.

why do you say this? This is not obvious for everyone. And like how this argument isn't sound. So basically a good editor will challenge like the logic and the way you present your argument and try to outline if there are any flaws. And again, as a referee, your role is a bit different as your referee, your role is to actually make sure that the science is accurate, that

everything is rigorous, that all the sources that should have been cited have been cited, and just give like basically a technical, a really thorough technical assessment of the paper. And I think doing these things is difficult and requires different levels of maturity. And you need to go through all these stages to actually be better at writing papers. And if you have skipped some of these stages, actually,

You won't be so good at writing papers. And you mentioned AI. There are a few reasons why I've never been keen on AI in publishing, in particular in peer review. Again, writing a peer review report is not easy. We are not taught properly how to do it. You get some insight from your PI, but we don't get proper...

training. Actually, many places we don't get proper training in writing papers to begin with. We don't get proper training in referring papers. We don't get proper training in editing papers. You learn it as you go. And if you're lucky and you have a PI who is teaching you this skill, it's good, but not everybody has. And what we see like now, and it's not easy, you know, you need to do and you get better with practice and

the more you referee, the better you become at being more objective but also pointing like you'd see like early career researchers would be very quick like, this is wrong and this is wrong and this is wrong. And someone more senior might be like, okay, well, what does it mean from a bigger picture's perspective, what can this lead to?

You see, again, different referring styles. And my problem with automating this is that we lose that diversity of thought, we lose that diversity of feedback, even though we don't like it very often, that type of feedback, we get something standard that we might or might not be able to trust. And so we lose like the real...

the really great results because sometimes when it's something like a real breakthrough, it's really hard to recognize and will be easy to dismiss. And it takes a leap of faith to actually believe in that new result. That's where you normally try, you know, when you have something different, you'd get different referees and the opinions would be quite spread out and be the role of the editor to try to see, to balance those and see what.

decision should be. And sometimes you get it wrong and sometimes you get it right because it's not, you know, we just don't know. And sometimes the paper might be amazing and sometimes it might turn out to be like, that wasn't actually so great. But doing that takes a lot of critical thinking and again, takes a bit of flair and takes a bit of a leap of faith, which are not qualities that you can easily automate into an AI peer review system.

And

Okay.

like Anderson said, if an AI editor received Bell's paper, would have probably discarded it because it was completely out of distribution. And we wouldn't be in a good place right now because at least we don't know how quantum mechanics would have evolved otherwise. But for this reason, I think that it is important and also because it's part

of the training of scientists, developing their critical thinking, of developing their appreciation of science, and developing their capacity of conveying their ideas, their theories to others. And you can

Mm-hmm.

only learn that by doing it.

I actually wanted to talk about the publication process later, but I think let's go at it now. and I w I would like to push back a bit on this, from a couple of dimensions. if we think about the whole publication process, there are obviously different stages, there are different parts, different processes involved. You already distinguished editors versus rev reviewers, for instance.

now on the reviewer side, I think it's a very good exercise to learn it well. And I agree with you basically in my own experience. It's it's totally treated as a sidestep. It's a waste of time, essentially. It's just costing your own time that you could work on in a kind of more pr kind of more extreme version of it. but I've seen tools and I work partly on it myself. maybe this is also representative more of the field of economics, because reviews are often

bad quality that you get from humans. That my feeling is actually tools, AI-based tools can help tremendously, in a sense of improving papers and also just, you know, almost like a first quality check if you want. It doesn't decide on the on the overall merit or in the the contribution on this kind of more fuzzy thing thing. Where where could this go? Which is a maybe we can talk about sec. But I see this, there's clear value in helping and supporting this process. How do you what's your point or what's your view on this?

think

there is, and there is certainly on the production side, on some layout side, again, as you say, we put this in a really good shape. And I think technically, how you produce the article and how you do, although I wouldn't discard the copy editing because what some human copy editors do,

I think is pretty amazing and the ability to spot inconsistencies that haven't been spotted by the editor, by the authors or by the referees, that's still amazing. However, that level of copy editing is very rare and I think AI can do a pretty good job there. Hard to tell if better or worse than a top level human copy editor. So certainly there's room there to improve.

What I'm particularly worried is that we're lazy as human beings and the more tools we have, the less we want to do. So the question is,

Mm-hmm.

how do you ensure the rigor and how do you ensure that we don't just like, I was checked by, I surely find I don't need to have another look at the proof.

I don't know how we go around that because we've seen there's a lot of recent research that actually shows that instead of making us more efficient in doing tasks so that we can do more creative things, actually AI tools make us do more for the lower quality.

I would actually want to see AI tools that also in the system in which they use that actually incentivize us human beings to do better quality jobs rather than incentivize us like to be lazy and that's okay. Where AI checked it. I don't need to do.

think the last point that you mentioned is super important. It's a it's a systematic question, an organization of processes essentially. Clearly if you just hand out a tool with no change, then it's the same thing as what happens in b in in the business world where established companies see tools. But clearly if you want to adopt these tools rightly, you also need to think about your organizational processes from first principles and might reorganize your company in a different way.

That's true. So like I think the incentive, we need to change the system of incentives so that actually we're incentivized to do quality originality and creativity rather than use AI to produce a lot of mediocre stuff because

Yes.

it doesn't benefit anyone. I think like my view and again, maybe unpopular. I think the way too many articles, we don't need all these articles.

I agree.

Thanks

The scientific

article evolved through history to this particular format that it has today because of the constraints of the medium it was produced and because it evolved like as a system for finding information basically. So we have abstract introduction and so on, references, methods, because it's evolved in that way. It wasn't always like that.

It's evolved in that way so we can find information very quickly because as a human in the past, I would need to go through many papers to find one thing I need. I need it. And I would read 20 papers to find a number that I needed on page three in table five of this article. And I needed to do that quickly. Right. So, OK, this is on the topic. Now we don't need to do that anymore. But the format has remained the same. And.

I think where AI could be really powerful would be in creating rigorous documentation because we focus on the end product that is the scientific article. And sometimes we neglect like the documenting the process that lead to those results. And this is someone

Mm.

who has a very lousy, had a very lousy notebook in research and didn't take, document the process carefully. And I think AI could help us.

a lot in actually documenting the process of research well, and we already have electronic lab books that have some AI assistance. And actually, for me, the ideal world would be where all the efforts are put in there and that you have an archived, cross-checked, findable reference of what the

process was and everything that you've done. And if you're a graduate student and you're sometimes sloppy, AI would actually remind you, hey, have you taken note of the temperature in this experiment or have you followed step five of this protocol? So then you do the science rigorously, which should help with the reproducibility crisis. Documentation exists there for the lab. You can share it. And it also helps you.

create metadata for your data sets, write code that is good and like lots of academic codes that used to be written in the past, document everything you do. And you have this information which exists in the lab and then with AI you can repackage it, say we want to share some of this knowledge with collaborators or with the funders to show like what we've done. Could be part of the review process. But then like I feel like

Or it could be part of the review process, right? To shortcut it, yeah.

The articles themselves, they should be written by people, for people, where they interpret this work and explain what they did it and explain how is it important and try to convey the information. And then you'd write, document very well the process, document very well the outcomes, but the interpretation and analysis, that takes a lot of critical thinking and leads to your own understanding of the subject.

And that you should do because otherwise we're not doing science if we don't do exactly. That's the fun part of it. Right. So let I do the boring stuff and we can do like the really interesting bit. And then that article shouldn't even look like, you know, traditional form of the article could be an essay, could be a different form, could be like a podcast like we have today. You know, again, we don't need to be.

frozen in a medium that was there, like in the format that was there for very specific reasons. And to me that would be ideal. Well, again, as a system incentive, I don't think we have the incentives to go there.

Can I, so you touched before on this question of, there's judgment basically is the aspect where we find AI quite disappointing. And you're not the first one to say that. I've talked to people for instance, who, know, assign grants, like public grants. So they can't come on the podcast because there's politics around them. But there's obviously like a usefulness to a point when it comes to like technical review.

to a grand proposal. And then this, like the judgment stage is where AI kind of seems disappointed. And I'm still trying to figure out why, right? I'm still trying to figure out why humans seem to have this skill of distinguishing, say maybe something genius from something crazy, right? Because usually what makes, imagine, like a really good paper, like say top tier conference is something that moves the priors, right? Something that

is out of distribution and somehow a really skilled, tasteful editor will have some instinct about, know, what is out of distribution because it's silly and what is genius. And I wonder if you have a mental model of like, what is this capability that we have? Where does it come from?

my feeling is that this is partly because of tacit knowledge. So there's so much more than what is captured in papers, specific fields, like all the people you know.

and all the conversations we had at the conferences and all these additional things that you've seen in the lab and you've done. And you know, like for me as an editor, the big part of the process was that I would go and visit labs and talk to people, go to conferences

Mm.

and talk to people. I know a lot. And now, know, when the paper comes, it's just not that paper that I assess in isolation of everything else. That is in a web of information that I have.

about who's working on what, which groups are trying what. And I've heard at conference of this coffee break, this group is trying exactly this, but actually they haven't managed. And you know, there's all these things that,

Mmm.

and it's often like when something new comes out, you know, it's been in the air, so to say, for a bit. So people are trying to articulate something, they feel they're up to something. And you know that.

if you know if you have this social you're embedded in the social network of that community and if you hear the conversations and this is something that is not recorded anywhere so you don't have access unless you're part of the community and you're part of the network so you know that and again it can work the other way around because

you know, community doesn't believe in this thing and everyone thinks like, this is this tasteful, this approach and it just goes against what we believe. And then again, this can influence you to like say, no, no, we won't publish this. And it's a brilliant thing. So, so, you know, being part of the community and part of the network and having access to this deep tacit knowledge is part of it and but can work in both ways. And the other thing is like,

AI, so I wrote about this, the way I see it, AI has extraordinary breadth, but people are much more narrower, but they have much depth in their fields. So where AI would been trained or say large parts of the corpus of scientific literature that I would never be able to read in 20 lifetimes, well, I know very specifically in my very narrow specialty,

I would know the experiments, how they're done, what people are doing, what people are talking about, what has been tried and what failed. And again, this record of failure doesn't exist anywhere but in the tacit knowledge of the community. And that depth and that knowledge of failure and that tacit knowledge actually allows people to know things that are hard for AI to know at this stage.

And again, I'm not saying that's impossible. I'm just saying at this stage.

And I do think that as we use AI more like say to what I was suggesting to help us with the lab notes, as we assisted in the experiments and it gains more and more knowledge about the process of doing science, we'd be more embedded in a sense within the community and we'll have more and more of this understanding. So I saw a lot like the experiments where like they used to teach AI like how you do like all the chemistry experiments and what you do like physically.

when you prepare like follow protocols. So at the time being, AI doesn't have this knowledge.

We had another guest coming from a very different field, the filmmaker, was telling us that if he was going to try and teach an AI to have taste in an area, he would start by cutting the internet connection and making sure the AI doesn't look at too many things. Basically, he was kind of making this point that you kind of need to not know certain things to be able to focus on one aspect. Is that compatible with your mental model as well?

I

think to be a good scientist, need to be in a sweet spot between depth and breadth. If you're too narrow and only know your field in however much detail, you will make progress. You won't necessarily be creative or make these kind of breakthroughs that influence around, or it's less likely. And then again, if you're too broad and you're too much of a generalist,

you would not have the depth to appreciate those really good questions or really promising problems that you should tackle. So this is a very good spot between breadth and depth where creativity works best. So I would say like you do need to be deep into something and then you need to be able to look around. if you look at the biggest like, you know, like

many of the breakthroughs in science, it can by connecting different ideas.

Can I interject something here? This if you think about it like across deep but also vertical, it seems to me current AI is actually you know very good at helping oneself to position in either direction. Right? You I mean s in some sense, if you if you have a certain level of depth, but you want to get a certain breadth, it's much way it's way easier basically to get a broad understanding and basically more of things also translated

Yes.

basically from different fields into your own. and

Yes,

actually there's a good name for this, it's called the T-shaped scientist that's broad

Okay.

and deep. I think the depth you need to get by doing. I don't think there's a way around of getting the depth, you have to do it. But the breadth, yes, AI can give you that breadth and it's difficult to get the breadth, but the depth you still need to work. And this is what brings me to my earlier point.

I think if we just give an AI tools, we just be lazy. And that's what we see in the results, which are not great. The way we need to use AI is to challenge ourselves. So it gives me a big, a new ability so that I actually challenge myself in the other, on the other dimension to bring out something new. So, you know, like.

I will work hard and I'll specialize in an area and I know a lot of things about that area. And then I use AI to broaden around and having that will help me actually formulate new problems, ask new questions and do things that I couldn't do before. But if I just, do, we need to put work in. That's what I'm trying to say. If we as human beings don't put work in.

the results would be mediocre. So I think AI is great tool, but the problem is with us and how we use it.

Before this call, you had shared some very interesting links with us, including one to an article that talks about the exact title is Unintended Consequences of LLM as a Labor-Augumenting Technology in Science. And this article is making a very interesting point that switching from one topic to another is not much cheaper than it used to be. Like there's a change in opportunity cost in a way.

And as a result, it's becoming unclear whether people will be persistent in that research. It's less clear whether people are going to push deeper, I guess, or longer, however you want to call it. Do you want to maybe tell us more about this?

So I think there are lots of interesting studies that are done these days. And this study you mentioned, just shows like we're doing more, less well. And I

Mm-hmm.

think the problem is not with the tool, as Philip said earlier, it's with the incentive. Because if the incentive is to publish volume of papers, why should I spend time to make them good papers?

I'll just use AI to make more papers. And this is where I'm saying like, optimising for utilitarian short term goals is not great, as we know, because it just drives you in some local minima. But it's the same thing if you assess science by only one metric, like, I don't know, number of citations or whatever.

because this just makes people play the game naturally and try to optimize only for that metric. Whereas if you assess on less, a bit ill-defined,

Mm.

broader spectrum of measures, and there's some uncertainty in the weighting of these measures, it's not random, it's not completely subjective, but it's along different axes, and there's some uncertainty in the weights of these, all these...

access, but then people have to have more interesting strategies and they can optimize for a region rather than a point. But that's again, we're going back to the system is the system that we need to adjust the system of incentives so that we use these AI tools correctly. Because if we don't change the system as Philip said, and we just add the tools, the results will not be great and we can already see the beginnings of that.

Yeah, you're touching on, I guess, some of what you're saying is relevant to LLMs and some is independent of it, right? I guess even before LLMs, had

Yes.

trends in science optimizing for H index or whatever, which would potentially have these negative impacts. But

Yes.

on the positive side, some of the best universities have kind of changed their rules, right? They would find, for when they interview someone for a professorship,

Instead of looking at a number, they would ask, okay, you can only present your three best papers, right? Or they would make sure that top quality is kind of revalued. But I would say independently of that, there's still this question of, and I think that's what this specific article was saying, is that even if your goal is to have one day the best possible paper in your career, and you're only going to be judged on the deathbed on this one paper, there's still this question, how long are you pursuing an ID?

And how quickly do you just shift between ideas? And AI is this really, it's almost kind of an addictive experience where because you can learn so quickly about a new topic, because you can play so quickly with new ideas, it's easy for someone to get into this bad habit of just working on something

Yes, because

else every day.

perseverance is really, important in science and pursuing goal like almost single-mindedly. So we see

Mm-hmm.

like all of the most successful scientists, maybe this is not good in your family life, but like they are very,

Yeah.

very almost obsessive in one direction and persevere, persevere again, all odds.

into one direction and that is important. that's resilience and perseverance are really, important. And again, is AI helping us do that or is actually pushing us in a different direction? I think, again, it's the thing like breadth and depth and you do need the depth and you do need to work hard, like to dig, dig, dig, dig, dig down in one direction to get results.

But I also think with AI, there might be other opportunities. now we're just at the beginning. We think LLMs are the end It's just a step as we don't know what's coming next, what technology is coming next. That will change things even further. I think it's an opportunity also to change, as you said, they were already...

drivers of the system that weren't working very well. maybe it is an opportunity to change things, but I'm also worried it just pushed it, although it is an opportunity for interdisciplinarity or true transdisciplinarity, actually, it's also pushing in the opposite direction. what I see, and with that, I want to come back to Phil Anderson. What I see is reductionism. So reductionism is great in science, right?

because you break a complex problem into little bits and pieces and solve them and makes it tractable. But reductionism doesn't imply constructivism. Once you know these

Mm.

little bits and pieces, it doesn't mean you can rebuild the whole. And this is what Phil Anderson said actually in his essay, More is Different in, I think it was in the 1972 or so. And that was really, really interesting observation.

and also showed like at each level of complexity have new emergent phenomena. And you'd think like, you know, this is very well accepted, especially in physics community and then complexity community these days. But what we see actually the way AI is used is like to enforce reductionism because you see like AI for mathematics, AI for physics, AI for biology, AI for materials discovery. And this is like really

a strong form of reductionism and even stronger is this kind of end-to-end science that is like, okay, automate hypothesis generation, automate the experiment doing, automate this, and it's a series of blocks that you do, and then you automate the whole process of science. And I really want to mention this editorial in science that I really enjoyed from this year.

and it's called Progression Without Progress by Julio Ottino and Brian Uzzi. And it says something that I found like, it's like following on ideas from Karl Popper, but says like, science is not a clockwork, science is a cloud. And you can optimize a clockwork, but you can't really optimize a cloud. And also like, the thing is that when you try to optimize the little bits and pieces of this complex system,

What happens is that the whole doesn't work better. Sometimes it gets worse. And my worry with AI is that it just leads to like very extreme forms of reductionism. Contrary to the fact that it is an extraordinary good tool to have a more constructivism approach. So

Mm-hmm.

again, the thing is, why is it favoring reductionism? Because reductionism is

easier to tackle and is better in the system of incentives that we have. Constructivism is more expensive, more risky, so it's not incentivized in the current system that we have. if there's one overarching thing of what I've been talking about, I think AI has great potential because it's transdisciplinary. But we are using it perhaps

in a bit too narrow way without seeing the bigger picture. And we're also too much focused on utilitarian and short-term goals rather than trying to see, this is a great tool that we have, integrative tool by design, know, what are we going to do next? What is the bigger picture? How can we use it into more integrative constructiv...

constructive approach that actually brings disciplines together rather than pushing them apart further. So we can't change the system but we can start thinking about how could we change some of the incentives.

Maybe

in that, it's the kind of effort you're alluding to is something I'm lucky to have worked on quite recently. So the, in the, the AI space, there's a lot of talk about recursive self-improvement, right? So everyone is kind of trying to get AI to make AI research. And the, the typical pattern they use is something you alluded to, which is, okay, you have one.

agent that comes up with ideas. have another one who runs some experiments. You have another one who take notes and then you come up with the next hypothesis and something. And on the one hand, you could say this is a bit rigid and simplistic and research is much more messy. know, there's much more

Yes.

to research than just coming up with hypothesis. But at the same time, the AI is in a way much more flexible than previous form of

digital automation, It is actually surprisingly on the surface. It can look much more creative and fluid. And so that's where the optimism sometimes comes from, right? Is that people think, actually, this can actually come with good ideas. And then the third level of like, least I'm just sharing my journey of discovering the limit. And then the third observation I had is that actually, even though AI is quite flexible,

there's kind of a finite number of good ideas it will come up with. And after a few experiments, it's just going in circle. Like it's not really a... I guess I'm curious of in practice, what do you think are the real blockers? Like why would AI not manage to come up with more ideas than people?

Well, I think because there's a lot of randomness, but it's not just throwing dice type of randomness. It's again, like a lot of randomness guided by intuition. And then this intuition that leads us to taste and leads to this ill-defined phenomenon, ill-defined quantities. You know a lot of things that are not digitized.

in that AI doesn't know and you talk to a lot of people and they say things that are not necessarily connected to that. So we live in a real world, we interact with the social network, we're exposed to a lot of stimuli that are not there. And all the ideas in science have come from connections to a completely different domain, just because that scientist was interested in music.

or had an interesting Indian philosophy, they would come up with really unusual connections. so, I think, for me, think it's just human beings have all these other stimuli that don't get in the agent, like all these random things that we are exposed to. And we also have our own biases and interests in particular areas that we will try to...

no more and then try to make connections between those areas and the areas we're working on. And ultimately we play. We're playful beings. again, like you've noticed by now that I have an interest in the history of science. Richard Feynman's Nobel Prize was actually the origins of the idea was him watching people spinning plates in the cafeteria at Caltech.

And that made him think, why did the logo on the plate show the pattern? that led him to... It was like a completely silly thing to think about. And people around him were like, why are you interested in this problem? But thinking of this problem being playful and asking this question led him to advances that ultimately earned him an overpriced. So, you know...

AI doesn't yet have that capability of playing and asking random questions and being interested in apparently niche subjects. So that richness of randomness and that capability of playing are not there yet.

think

that I think this is actually ev also evidenced by the fact that different people right now can bring very different things out of the current AI, right? Which is that's the same model. But it's in the people that basically bring this model to produce, let's say, a scientific paper on a high level or on a very sloppy level, right? And I think your point before is also this is we discussed it at length already, but just alluding back to it, it's really the question of how do we organize this in a in an overall fashion. Because

You know, I think it's good and y you if you can as a graduate student, you can basically almost leverage this and be have your own lab at some level, on a small level, which is not fully. But then you again you're part of a breaker process. And the question is how is the how do we organize? And I mean, and you know, humans this is why we have institutions, right? We we invent them to tackle these problems of combina combining things and bringing bringing things together.

I think

so. How do we organize ourselves in this new world? How do we make sure the incentives drive us into right directions? But also, how do we train not only the younger people, but how do we train ourselves to use these tools and not actually use them in a lazy way? And I think my example is you can always go and use an LLM and ask a generic question, you get a generic answer.

my approach is, is this worth the energy and water consumed to get to generic answer? How about I take a bit of time and I think carefully how I'll ask the question so that the question is more specific and actually the result is worth the energy consumed because this provides a meaningful result. But then, you know, that takes a lot of, puts the effort on me. And again, as I said, like,

Do we want as human beings to do that? Or are we incentivized to do that?

That's the question really.

That was it was fantastic. We have already taken up a lot of your time. Maybe let us end on a on a short answer from you on a on a very difficult question, which I know is a contradiction, but anyway, let's give it a try. If you had the opportunity to change one thing and one thing only, in the current sausage factory of how science is produced, what would it be that you change?

That is very, very difficult question indeed. I don't think one thing would help because again, going back to...

You're pushing it to the next constraint. Yeah.

No, because of the reason as I said earlier, like optimizing one thing doesn't mean on average everything will become better. So this is a complex system. One thing might not be enough and one thing I might think that if changing it will do better and actually do worse.

So perhaps let me put it in a different way. What I would like people to think more carefully about that, you know, this is a very complex system. Or even if we know the bits, we don't know the emergent behavior of this system. So we have, it doesn't mean we can't know it, but it means we need to be very careful in how we model that. And quick solutions that I'll fix this bit here and therefore we'll have an amazing outcome.

it's increasingly clear that they don't really work.

That's a very good point to end. Thank you very much, Iulia.

Thank you very much.

Thank you so much.

Thank you.