← Back to episodes

Markus J. Buehler

McAfee Professor of Engineering at MIT and member of the National Academy of Engineering; leads the Laboratory for Atomistic and Molecular Mechanics and composes music from the structure of matter.

Auto-generated transcript — lightly formatted, may contain errors.

Markus J Buehler

The arc of the sort of development, you create a music from a protein or maybe a protein from an old composition before we knew about proteins. what is the sort of the the cross talk we can generate? So we have this piece that deep aria, which which I'm I think is quite interesting. So uses Bach's compositions and makes it into a protein and actually recursively sort of has a talk between the protein and the music, sort of Bach style music, and it creates this conversation between things that were never possible.

So that's something I think you can do with music, of course. You can create a virtual reality in which you can have a protein talk to billions of years of evolution to through billions of years of evolution to something like Bach. And we're doing this today and making the connections, like we said earlier with category theory. Now something new comes about, which we've never seen before. And then we can study that and it creates a wonder and maybe surprise and or maybe

shock.

Philipp

Welcome to Tastebench. I'm Philip Zahn. My co-host is Bruno Marnette And this is a podcast about Taste and the creative side of AI. Today's guest is Markus Buehler professor at MIT and co-founder of Unreasonable Labs. His research connects material science with chemistry and biology. In recent years, he has focused on the automation of discovery and design processes of new materials using AI.

Markus it's an absolute pleasure to have you on the podcast. we will talk about your work crossing into AI and what you do when it comes to automating science, which is a big topic as far as I know. But before that, I think it would be good to start with you personally and your story into research, maybe starting with a PhD. What was your PhD about? What was your initial research focus?

Markus J Buehler

Yeah, so my PhD was actually at Max Planck Institute in Stuttgart. It was called Metals Research at the time. It's now called MPI for intelligent systems, which actually turned out turned out to be one of the AI

Philipp

Yeah.

Markus J Buehler

hubs in in Europe. yeah, but back then I was working on basically materials modeling, first principles modeling of materials, particularly looking at fracture and defamation. And you know, we were looking at sort of basic research and how

metals behave, ceramics. I got involved at the time actually in biological materials. We had a group there that became interested in sort of applying material science to biology. we were studying geckos and insects and all that stuff and so that's what that was at the tail end of my PhD. But my PhD yeah was mainly on molecular modeling of fracture.

Philipp

in private conversation you said you actually started with work on material science and category theory, which I mean I'm familiar with category theory, but I think for the average person who knows category theory, that's a wide stretch. so maybe can you give us a bit on this arc? How you started, what were the key things that you were working on? And

how this evolved also into into AI.

Markus J Buehler

Yeah, so as after my PhD in Stuttgart, I went to became a postdoc at Caltech. And at that time I was working on mainly sort of more force field development, first principle simulations with Bill Goddard, who's a who's a chemist. And then shortly after I moved to MIT as a faculty, and you know, when I started my work at MIT, I sort of developed a program really around biological materials and biology, in combination with engineering. And and of course

the the being able to simulate molecules and atoms and structures across different scales allowed me to simulate in you know in the morning spider silk, in the afternoon silicon. And maybe at night a little bit of metal metal alloy design, right? So I had sort of this capability of doing, you know, multi-scale predictive modeling through many different systems. And of course, as a theoretician I could

begin to make connections. And this is actually how we got involved in category theory where we ask questions of I'm studying all these different materials like proteins, spider silk, collagen, metals, ceramics, adhesives, all these things we talked about earlier. Geckos, right? how do we actually formalize this process of drawing conclusions and sort of lateral connections? And of course in science we do this all the time. You know, as scientists we, we have sort of this insight. Yes, I can study

I can further try to figure out how spider silk works by maybe applying things I've scientists have learned about metals. And maybe I can even do multi-scale modeling that has been applied in ceramics to the system. So we kind of make these connections. So yeah, a couple years in at MIT, I became really fascinated in with this question to formalize this process. And this is how we got involved in category theories. Actually, David and I met sort of in the Infinite Corridor coffee shop area here at MIT.

Philipp

Classic.

Markus J Buehler

And

we started talking, some students got involved and and we were quite excited actually, sharing this this interest. We have different tools. And that's how we got involved in in this really. And this actually sort of became a really major effort, with in fact actually very little attention from the community. People didn't really get it, maybe, or not get it, but they didn't really appreciate maybe the importance of this. and part of the reason was also I remember giving talks about it and they Well, how are you ever gonna scale this? And I said,

I have no idea because it was yeah.

Philipp

deadly sin.

Markus J Buehler

Yeah, yeah. I mean it was basically pen and paper exercise and then it would take sort of extreme effort, intellectual effort by humans, right, for to actually build these categories and being able to map connections and build we we call them OLOGs at the time of graph representations of categories with David. And yeah, I mean we we couldn't really scale this. And so this is actually how sort of this project sort of got stalled a little bit back in 2010, eleven, twelve. I've done some really interesting work.

in finding formal analogies between spider silk and music and language and different material systems. And and really the it was driven sort of intellectually by me because I wanted to sort of find a much more condensed mathematical description of what I'm learning. So if you you know run a simulation and I would run simulations with billions of atoms, sort of I figured yes, I mean, if tell another scientist or if I give a talk or maybe I teach a class.

I'm not literally presenting to my students the positions of every atom, right? Or my my audience. I'm actually talking about principles and mechanisms. So what is actually sort of the invariant property of my system to you know, the regardless of which yeah,

Philipp

Building blocks. Yeah.

Markus J Buehler

and and and this actually became sort of the the driving cause of course for category theory is is a perfect way of sort of creating relational descriptions of things. and I became really interested in automating this, but yeah, again, it was really hard to do back then.

And this this later on led us into AI techniques actually to when I realized that actually we can build these graphs using machines and we can begin to sort of not only mine the data but actually create abstractions of really complex relationships. And this sort of led into the more recent work, but we'll talk probably more about that later. But

Philipp

Yes.

Markus J Buehler

yeah, that was the origin. Yeah.

Philipp

Can I ask, you you basically said, Well, that's all that's what we do in science all the time. I'm not sure this is happening all the time that people are crossing fields that much. I mean, also kind of vertically in the sense of trying to find theoretical foundations, but at the same time merging biology, then maybe chemistry, classical engineering, and then also music on the other hand. Can you say something about this? Because this is, I think, highly unusual from the outset. If you know something about category theory, again, I think it's not that.

that far fetched in some sense, but maybe you can give us a bit of insight how this happened actually.

Markus J Buehler

Yeah, I I've always been you know interested in in in sort of taking things apart and understanding how they were built. And and and I did this actually I've been working on music and sound and composition for a long time, you know, even before I worked on science. but I realized actually later, of course, it's a sort of very similar process intellectually for me. I I look at a system, whether it's maybe you know, biological material or maybe it's a piece of music, and I'm trying to figure out what is the algorithm that made this thing. Right and and so that's

dri what's what drives me. And I'm I'm interested in the the process by which these systems are created, right? So of what are the the algorithms that make something and why is it happening. In music I think it's interesting because there's obviously not an engineering objective necessarily, there's sort of a creative process. And actually the early work in category theory, materials and music was actually driven by the fact that well, if I'm a scientist and I follow sort of the cookbook, right? And that's what you're alluding to maybe

I'm doing chemistry the same way all the time. I'm especially once you become more senior, you get really set in your ways. the sort of if you're more junior, you're more maybe exploratory and you're not tainted as much, maybe. but I was curious if if maybe you know, science is of course driven by creativity, and if we're using some of the mechanisms in the you know artistic sort of side of the brain, if you wish, or the process by which art is created, there could be some

im input into discoveries that might be useful. So that was one one thing. But there's actually a there's also a deeper reason. really you know finding these sort of generative algorithms and also finding the origin of something like this. I mean I'm really curious if you know in a creative process, where does it come from? And we actually analyze sort of a yeah process of composition for example and performance or composition and performance, I mean you realize actually that there's thousands maybe

millions of decisions the human mind makes in this process. And I you, could have realized this when I'm trying to reverse engineer, how did I make something? I write music and you make many, many choices. You know, why do you make them? And it's a lot of times it's intuition, experience, it's sort of a maybe a composable structure of things you've learned and you're putting them together in new ways. But it is sort of a liberating side if you're working in art.

Because yeah, you're not bound by the laws of physics and you can explore a new space of ideas. And then you try to fit it back into the real world and say, well, now if I have this idea, this crazy idea of how I want to structure a material, how would I actually you know make sure this abides by the laws of physics? But then you're in a different space of in innovation. So that's kind of like how we we look at it. And I I've been very lucky at MIT. We have, of course, lots of students and programs and sort of facilities here where we

We actually really this process of art and science is is very intertwined here at MIT in many different places.

Philipp

Yes. Can I sorry, maybe one one one more quote from my side, Brun, then I I stop here. But I just because it's a quick one. I c we c I came across your your SoundCloud compositions basically, and they they have funny names in the sense that they they are like coto ensemble of protein one AKG G Venom predatory marine snails. So i I the the way you described it now is more like from music into science. but the titles actually suggest the other direction.

in some sense. Is there something to it or is it just basically

Markus J Buehler

Mm.

Philipp

a are you what what's the connection there? Is it just the naming or is it really something more more deeper?

Markus J Buehler

yeah, no, it goes both ways. w I have done a lot I so I do work in music that is just creative without any kind of scientific input I guess. I mean maybe my mind sometimes you know we we create things that are images of something we're trying to study you know, scientifically as well. But I do work just sort of purely creative side, but I also have done a lot of work where we we try to create representations of scientific objects like proteins in music. and that sort of

so fast is a bi-directional thing, you can try to create a an expression of a of a of a protein in music, but also convert music back into a material. And what are the rules to do this? And there's obviously a lossless compression, which is actually the least interesting one. The the the lossy translation is more interesting because you get a residual. So you converting let's say your music let's say you're converting a protein into music, and you also make artistic choices.

So now you're merging the structure, maybe the vibrations or the sequence of a protein as a constraint, right? And or as a

Philipp

Mm-hmm.

Markus J Buehler

basis, but you're also adding your own sort of thoughts. And now when you're looking at the product of this, is it actually an amalgamation of your of the creative process and the scientific structure? And if you repeat this a couple of times, I mean you can really sort of end up in really new spaces artistically as well. So that's something that I I really enjoy doing, actually, is is using

scientific structures or maybe frequencies or scales derived from proteins or from molecules or structures, spider webs and things like this. yeah, sort of as a way of of constraining the artistic choices you you you can and you're allowed to make. And that yeah, it's a it's a way of creating art, I guess. Or, you know, and and in a way also telling a story. And so this the titles, I mean there's many different pieces on there. I yeah, I like to use titles that reflect a lot of the scientific thinking I'm doing.

but yeah, some pieces are just basically you know also fully just artistic creations. but some of them still carry some scientific names because I'm always drawn to that. My mind kind of wanders into the directions of yeah, exactly. I mean you gotta think about like

Philipp

Yeah, you can't help yourself, basically.

Markus J Buehler

s swamps and you know the variations and and I mean there's a couple of pieces there. but yeah, I mean it's it's it's an interesting area. And you know, for me, you know, we've done lots of work with

the community of of artists, we've done performances, we've done I've done the stuff that I'm doing by myself in SoundCloud and other places. But yeah, it's an interesting and it's a it's an interesting space intellectually. And I think I I you can reach people about sort of science that you wouldn't otherwise reach, obviously. So that's one of the one of the reasons why I also enjoy doing this.

Bruno Marnette

You mentioned earlier that music in your mind was maybe a bit more free because you didn't have to apply the rule of physics, but I imagine there's a lot of rules as well in music, There's the rules of harmony and so on. the same way, any art form,

Markus J Buehler

Yeah, well, maybe.

Bruno Marnette

right? Every art form would have at least some implicit rules, which good artists tend to break from time to time.

Can you maybe tell more about this?

Markus J Buehler

Yes. Exactly. Yeah.

Well I mean I think you're I know what you're you're heading with us. I mean, exactly. So so they're rules, but the rules are meant to be broken in in art actually. And and and this is also true in in in science actually. You know, we we sort of push the boundaries of a theory, of a model that we understand and we break it, right? And then we figure out a new theory. And so I think in art we we're trying to show people the boundaries of what what is possible.

So if you're yeah, if you're using the sort of harmonic rules, let's say that you fight mathematically, you're gonna create very boring music, obviously, most of the time. So so we're trying to break the rules, but you make the choice of when to do that and how to do that. and and so this is you know, sort of a a j general principle. And I think art really is sort of at that point, or music you know becomes a generative algorithm by itself. So you listen to things and you, you know, every time

you experience something new. it shows you something new, right? I mean it might show you a new perspective. it might, you know, make you wonder like you listen to it and you might say, well, this doesn't actually sound like real music. Well, it's because it's you know based on molecules and it it's designed or created to break those rules, right? And so so those are the kind of conversations. Or you can talk about what is the

The arc of the sort of development, you create a music from a protein or maybe a protein from an old composition before we knew about proteins. what is the sort of the the cross talk we can generate? So we have this piece that deep aria, which which I'm I think is quite interesting. So uses Bach's compositions and makes it into a protein and actually recursively sort of has a talk between the protein and the music, sort of Bach style music, and it creates this conversation between things that were never possible.

So that's something I think you can do with music, of course. You can create a a virtual reality in which you can have a protein talk to billions of years of evolution to through billions of years of evolution to something like Bach. And we're doing this today and making the connections, like we said earlier with category theory. Now something new comes about, which we've never seen before. And then we can study that and it creates a wonder and maybe surprise and or maybe maybe

shock. There's a lot of art, right? And it's

Bruno Marnette

Mm-hmm.

Philipp

Yeah.

Markus J Buehler

it's it's it's shocking, right? So yeah, those are some of the themes I think you can see. But yeah, it's definitely a play with perfection, imperfection. there's a draw I think we have to this. you know, artists I think are drawn to creating break the symmetry. You know, they they don't want to create per a a perfect structure. We we want to create something that that is designed, is deliberate, has an objective and intent.

yeah, so to do this you gotta play with the building blocks, which are not atoms in this case, like in the scientific world, they're tones and rules like you say, and and you decide when to break them, sort of how we push the boundary.

Bruno Marnette

When preparing for this, I came across your paper on selective imperfection, which sounds very close to what we were just discussing. One thing I would be curious to hear your thoughts of is something I often hear about music, for instance, is that, for instance, what makes Beethoven such a genatis is that every note creates surprise, but at the same time, feeling of of rightness. there's something like the next note is

not expected, it creates tension, all these notions that could possibly be explained in terms of probabilities or relatively easy perhaps to formalize. But then there's the other side of the equation, which is it's surprising but right. And I find the word right kind of harder to maybe formalize. it's something, I guess when you talk about imperfection, the flip side of it is possibly perfection or rightness.

Markus J Buehler

But

Bruno Marnette

I was curious if you could maybe expand on

Markus J Buehler

Yeah.

Bruno Marnette

this work that you've done and how this might give us a clue of how to define rightness.

Markus J Buehler

Yeah.

So I think the way we sing it, it's it's one of these attempts actually, like I mentioned earlier, to find the the algorithm that explains stuff. In this case, maybe music and compositions and how we use it. in a sense, we we create these these structures. and the way we're kind of have the journey laid out in the paper, you know, is kind going from like white noise, like all frequencies, and then you're allowing only certain frequencies

but then within all the possible frequencies, you're gonna pick a few that create some maybe tension. And then to create new structure, you you have to create something that is not self-similar. I mean that's really the the biggest message in the paper. I mean, if you create self-similarity, you create actually very boring and predictable outcomes. But if you create something that has contrast, right? So maybe one level is organized, another level is more chaotic and you know, distributed.

you're gonna be able create different palettes of sound at different levels. So the level of the individual note, the individual scale, the entire piece. So like in Beethoven or you know music like this, I think you're evolving through different modes or scales in a piece. And this you know creates this sense of exploring or painting something as a palette at different scales. Selective imperfection is sort of the idea that you you choose we choose, I mean as humans or

artists we choose certain type of structures that give us this palette to maximize what we can paint with. And in sound, I think we can actually see this mathematically. there's a you can compute all the different scales, you can compute all the different you know, how ordered they are and how many possibilities you have. And and you can kind of actually sort back calculate a little bit from observing what people have done.

what were the choices we've made without knowing about the theory, and then ask the question, what might be the theory? Yeah, but it's it's like in the in the world of materials you would have composites and you basically create a imagine you take a and this is sort of the analogy to the sound, you know, you you create you have an average material, but of course materials aren't just average. They actually are in average maybe the same property, but you kind of push you in your background you have a some sort of a

the composite maybe I don't know the hexagonal structures, but you know, you have kind of different colors, different stiffnesses. So the average stiffness might be the same, but you you kind of divide the world up into something very soft and something very stiff and maybe even interface. and if you do this you create entirely new properties, even though sort of the the average con constituent is always the same. So same I think is true in music as well. you're gonna have notes but by restricting what you use

you create this this expressiveness. And and that is going from noise into choices, selections. And you do this repeatedly, of course, at different scales, all the way from you know the individual sound of an instrument, right? An instrument that's just based on harmonics would sound pretty boring.

Bruno Marnette

Mm-hmm.

Markus J Buehler

you know, guitar sounds interesting because you have not just the you know predicted overtones, you actually have, you know, things that don't quite match, right, with the

perfect you know spectrum. And it goes on to the scales and it goes on to sort of the evolution if you have modal transitions in a piece and and rhythms of course and all that. And it becomes a pretty complex space, you know, which is mathematically sort of you know describable in a sense, but but of course there's much more than the math and there's also the choices you make within all these possibilities, which are really the interesting things that are that are at play.

Philipp

Can I interject the question here on this paper as well? Because one s one thing that struck me when reading it, or parts of it actually, is the notion of time and different time scales. time scales and how to kind of synchronize them in specifically in hierarchical organization is a big problem. It's a challenge in, you know, biology, but it's also in social science, in economics, organizational evil. It's a big question. and you make this kind of side remark almost that, you know, it music has this ability to kind of display it. Maybe I'm

depicting it the wrong way, but giving us access to yeah. Can you

Markus J Buehler

Yeah, that's correct. Yeah, so yeah, but yeah. Right.

Philipp

comment on this? Because in contrast to let's say a painting or other form arts

forms

Markus J Buehler

Right.

Philipp

of art, the time doesn't play that much of a

Markus J Buehler

Red.

Philipp

role. But in music it's essential.

Markus J Buehler

Yeah, yeah. So it's about the yeah, so it's actually it's a one of the very important points actually. Yeah, it's it's a small part in the paper, but it is actually one of the most important insights actually. And and it's about finitude or the experience. So in what I'm sort of talking about here is that if you I mean, we just talked about all the different theories and possibilities, if you were to lay them all out, right, you you might not actually experience the tension, right? You mentioned right Beethoven's sort of different notes.

So it's about, what is the potential, what you can create, and how do you actually lay it out? And the the rollout of the experience is really what makes music the the thing it is. and it's also true in discovery. So, you know, I think in scientific discovery you can imagine a world in which, you know, if everything you could know is known to you, you cannot experience obviously the process of discovery. and so we have to the the very human experience of discovery is in fact that we're

Breaking what we believe to be the case about the world. So in music, you play notes and you expect, yes, this is gonna continue in this scale. But actually, what happens is we're breaking that rule. Now we're moving into a different scale mode. maybe we're using totally different instruments, we're creating a connection, we're creating something different. The only way to sort of go through this is to experience. You have to live it, the finitude. So so if you don't if you would have infinite time and infinite possibilities in parallel.

You could never experience it. So that's something that we we think is also really a a c part of of creativity. Because it is this process of when the sort of the experience happens, the creative spark or the sort of the process by which you you change something. And and of course, as I mentioned earlier, I mean art really has this role in you know, good art will change people's perception about something. You know, there might be

Yeah, it might open up something new, it might shock them, it might make them look at something differently, and that is the sort of the impact there. it can only be experienced. And good art will generate this over and over again, right? You can listen to it or read it a book or many, many times, and you'll you'll learn something new every time, basically, right? So that's sort of the the experience you get. So the as a as a cre as a an algorithm, if you wish, you and and I don't think there's a sort of an objective to this, but

but you can feel it when you make it, that there there's certain things you create that have this this quality to them or others have created things of course like this. and they create this experience o over and over. And yeah, music is sort of this this I think an interesting medium in which this is built in, like you said, built into the the performance of it. there's no way to cheat, right? You cannot listen to music in parallel. I mean, even though you could listen to all the notes in parallel, but yeah, no.

Philipp

Yeah, you would miss it. Yeah.

Markus J Buehler

There's this axis. You

what you listen, yeah. And and I and I think and this is really connected to discovery and creativity. So I we have a lot of discussions on machine creativity. You know, what does it mean for machines? Can they be creative? what does it mean for us to be creative? And it's really about this experience. It's the moment when something changes. So you you create something new that that has never been seen before, has never been seen to be possible. So yeah, you mentioned probability. So it's not something that's likely, but it is actually happening. And once it happens.

And it is real, you say this is really cool. I I now know that you know, at the nano level we don't just have classical physics, there's something else going on. And it's also irreversible, right? It it can never be unseen.

Bruno Marnette

One thing I, so you mentioned music is something you would listen to several times, right? You could enjoy it at different time, every time you experience it. Likewise, kids can read the same book 10 times, 20 times. There's really something about this kind of formative process of like forming your taste, I think, when you consume this kind of the artistic content in general. But if you look at science, think statistically people are less likely to read research papers 10 times.

And I guess I'm curious, what's your mental model? Why wouldn't you read a scientific paper 20 times in the same way you read the book 20 times?

Markus J Buehler

Yeah, you

you could, I guess. But yeah, I mean it's it's probably more you know, it's probably maybe something it's a good question. but I mean it's maybe has something to do with the I mean I guess good science will actually have a very long trail of impact. So so in other words, if y if you have a paper that just basically makes an incremental contribution, yeah, you're not gonna read it many times. But I'm I'm sure, I mean you can read

the original papers on quantum mechanics, or actually, especially if you read the initial thought process. So actually I'm teaching a class right now on molecular modeling at my team for grad students. And we're actually going back to some of the early papers when they did f the first Monte Carlo simulations back in the 1950s. And it's incredibly interesting. I actually I like reading it over and over again because, you know, I mean, maybe five years ago, before AI could write simulation code or c simulations.

Has a different meaning. Now I'm reading it actually this morning and I'm like, wow, this is pretty incredible because you know it has a totally new context. So it does actually have in in in really sort of powerful advances, things that will change. and they will look different, you know, at different times in your life, in your in your scientific life. but it depends on the and that's how I would say is the quality of good work actually, is that it's something that is so foundational that it has a lot of connections, even much later. you know, and

Yeah, not all music is great, right? And not all art is great. it it's it's some artists

Bruno Marnette

You

Markus J Buehler

you look at it one time and you're never gonna be interested, you never remember it. But there's something you remember. And it's about

Bruno Marnette

Yes, yes.

Markus J Buehler

these experiences that change you. But of course, aside from this, I think there are clearly, you know, in in music, I mean one of the things in art we we don't really understand there's no theory really for the wandering experience. This this is the

I think the the thing that really we we don't very human experience, we're very drawn to this. I think in n natively in our, you know, very sort of baseline of of our of our existence. I think we we have sort of emotional connections to to things and yeah, there's no theory. It's not like just math or something logical. There's an illogical consistency to this. which makes it probably a little bit more

emotionally charged and and and so

Bruno Marnette

Mm-hmm.

Markus J Buehler

that maybe is why I mean art has maybe sometimes sort of more this experiential quality to it. But yeah, like I said, I mean as a scientist I I do actually enjoy actually reading old papers and and looking at old ideas. And I I especially now, you know, we can go back to early work and sort of look at a new light and make new connections. there's actually a lot of richness in a lot of early work.

Bruno Marnette

I have

a, let me just follow up on this. We had a guest recently who comes more from humanities and he was telling us the, so he was doing research on the how people form opinion. he talks to a neuroscientist about this question, for instance, and what we learned from putting people into a SMRI. And very often the impression he got, and actually scientists told him explicitly, is that

modern science, modern medicine often kind of falls short of explaining things which have been explained by literature much better, like across, you know, centuries of humanities in a way, right? So there could be an argument to say that, all these kind of human emotions actually are, you know, there is a theory for it. It's just kind of taking a different form, know, less symbols, more words, more literature, more poetry, and so on.

Markus J Buehler

Yes. Yes. Yes.

Bruno Marnette

Is that essentially a good

Markus J Buehler

Yeah, yeah.

Bruno Marnette

Would

you say it can be a theory? just the theory

is expressed

Markus J Buehler

Yes.

Bruno Marnette

in different language?

Markus J Buehler

Yeah, it's it's it's a different yeah representation, I guess. Exactly. No, and this is actually what I mentioned earlier about creativity. we are using sort of in the in the category theory line of work, we have made that argument that that if you wanna understand things that are, you know, maybe really complex, there might be an easier representation in music, for example. For us to understand, you know, build a model in a mind.

And yeah, you're right. this model is not gonna have equations. It's gonna be a model of maybe sound or composition or an emotion. And that actually is how we thought also sort of the early work actually was sort of dreaming up the idea that maybe in in in human creativity you find solutions to problems that we haven't asked yet. Okay. Now the question

Bruno Marnette

Hmm.

Markus J Buehler

is how do we get the question, right, that we've been asking.

But it's the having solutions to problems. So it's it's like imagine you have this these books which give you all these math solutions, but you don't know the problem, right? So this is actually how we early we described we some also some of the early working category theory of motivation for this. and then the question is, yeah, how can we find out what we're actually solving for? And and maybe there is there are solutions to problems there? Yeah, I totally subscribe to this. And I think

Probably the mind is able to operate if you we were talking about composition earlier a little bit, when I when I look at this from the lens of science and I actually try to reconstruct what I did and why I made choices, it is

Philipp

Okay.

Markus J Buehler

astounding to me actually. Because if I'm

Philipp

Yeah.

Markus J Buehler

writing a proof, I know exactly what I did and I can go back and tell everybody this is how I make this choice. or set up a simulation or whatever I do. In music I actually I I would not be able to s explain it.

And it's an intuition. But it is in level of complexity quite similar actually. And and I would say there's no difference intellectually, I think for us, humans are s really good in doing these things. So it's like, we have a certain spatial capability, we have things that we can do as humans, and then dogs have other things that they can do, right? And and other animals. and we have skills I think that are not yet tapped into in science that we we might be able to.

And would make us and maybe AI can help us with this actually. So this is, I think, the hope we all have is that we can make these connections more clearly and use some of the native human skills in doing things that a machine cannot yet do, right? Because a machine can do math, it's very organized encoding, it's very structured. but these choices, these intuitions, the sort of the reasoning across different domains, maybe, is something actually humans are very good at. so, yeah, absolutely. I totally subscribe to that. I'm I'm a big fan of this, and I think.

I I want the students also, the new generation of researchers to incorporate that and to think about this. And I think there's this golden age coming now where you can actually do this, even if you're not a musician or an artist, you you actually be able

Bruno Marnette

Yes.

Markus J Buehler

to do some of those things, yeah.

Philipp

there are so many points. we have a lot of notes, but I I keep stacking notes now basically of things we can branch off. This is super interesting. Can I maybe continuing on this this connection? So from the creativity and also the the the the paper on the music swarm that you wrote, which is kind of let me try to connect several things here. So one is in this paper, but also in the Sparks paper that is more about, you know, let's call it accelerating or automating science. there's an

A lot of emphasis on the architecture itself. So not just basically constructing it in a way that there is interaction, which is, you know, obviously also an important ingredient in other in in kind of industry applications of AI right now. We know this. maybe can you comment on this? And also if you take the residual, which is the the human touch that still is needed and maybe even mandatory or really, really good. Maybe can you comment on this a bit how you see your city's position here?

Markus J Buehler

Yeah, I think it's a trajectory through different stages of capabilities in AI. I mean, I think the early work like Sparks and some of the earlier models we've built, we basically, you know, kind of told the model how to do things. Like we had a coding agent, we had a you know, execution agent. Yeah, yeah. And and Sparks Yeah, yeah, and

Philipp

Yeah, every yeah, we remember this stage of yeah.

Markus J Buehler

and Sparks was actually an attempt to automate more of that. And so we had

It was meant to be open ended, right? So it does open ended research. So you have agents that don't just take the question that I'm providing as a human, I'm actually have the machine ask new questions and challenge the other agents. So it's based on that necessary architecture. but I am you know, I'm quite bothered by this actually, because you you don't want sort of this human driven choice i in in this in the machine. You know, we we have been building a lot more swarm based systems and and really, you know, autonomous systems that are much more

independent of the human design criterion. in fact, some of the later work, you know, like the, you know, the infinite system or science claw and then Swarm World we published recently are are sort of attempts to to take most of that out of the picture. There might still be human intuition in there and then maybe problems we're giving, but we really want the machine to to think about its own reasoning process, figure out how to how to solve a problem, correct itself and you know, you don't talk to humans of course. But

But do this more autonomously. Now, in the in the the protein swarm example, actually, or in the the the music swarm, I should say, that's what you mentioned, right? yeah, we the attempt there was sort of this to ask a question that is actually, especially for me, is very interesting. If I if I'm writing music, I have access to the whole thing. I mean, I can look at my whole score, I might listen to parts, but I'm listening to the whole piece, you know, many, many times. You you write a certain part and you spend like days of writing.

a certain section or transition and then you go out and you listen to the whole thing and you realize you totally messed it up. Okay, because now the whole thing is broken.

Bruno Marnette

Mm-hmm.

Philipp

Yes. Yeah.

Markus J Buehler

so you have this luxury of doing this. So I was curious if you had agents that aren't really aren't able to see the whole thing, they only see the neighboring notes, tones, maybe the neighboring bar. can something interesting come out of that and and and is there something that a machine can actually construct without sort of long range

perception and and and actually so this was sort of the nature of this experiment and and actually came out to be quite interesting. It turned out actually that it created much more unusual music than if you were to take the the language we use language models, which are really I think sort of general general intelligence format used in the in a sense. If you take a language model and say, write me this piece of music, it will write it.

But it will be like Bruno, you were saying earlier, it will be it you it follows all the rules basically. Because it's sort of Yeah, yeah.

Philipp

Ticking all the boxes, yeah.

Bruno Marnette

Yeah.

Markus J Buehler

And and it sounds really great the first time you listen to it. But it it will sound really not very interesting. So what I found actually, and this is an experience I had myself when I when I wrote the paper and did the work, is I had the Swarm make the music and I thought this sounds absolutely there's no structure, there's nothing in it. And I don't remember if I talked about it in the paper.

I didn't, but I think I did. And I actually ended up putting a rhythm track on on the on the on the on the thing that they that the swarm made and I and I thought just to help my ear. And it then it clicked. Yeah, it made sense. It's like wow. This machine created something that actually there's actually a structure in this music, and and and I can actually learn it. And so this is actually to me, it was the most important part of the experiment was that I myself was transformed by this. I I listened to

Bruno Marnette

Hmm.

Markus J Buehler

the machine composition from the

the the agent that saw everything. And I made immediate sense. I totally got it what it's trying to do, but it was sort of one one one one what once and done, right? Sort of the thing. And the the Swarm created something that really made me it challenged me. and and I and I ultimately understood what the machine was trying to tell me. So this actually, I used this in an example of a of a panel we had on machine creativity. As an example I said, you know, this is something I felt the machine was creative. It it created something that I

I hadn't seen it surprised me. Yeah. It

Bruno Marnette

Mm-hmm. Yeah.

Philipp

You were surprised by yeah.

Markus J Buehler

took me it took me intellectual energy to work through it and to actually understand it. So that that is something that is quite interesting. and there's always many different gradients you can use. And maybe that's not the perfect way of composing music, but it is definitely something that we we also found in the protein. So we had a follow-up paper on proteins with this similar idea. And we also found that if you compose proteins in this way, you also get much more creative solutions to proteins, as opposed to using

a a model that's been trained in all the existing proteins because then you fall back on all the existing biology. But if you use this protein swarm, the system explores r regions in the in the chemical space, biophysical space that just nature has never explored. And I find that very interesting. So we actually made some of those proteins in the lab and and and shown they actually they exist, they're real. But they're very different structurally than than what nature has invented. And that's something obviously

in discovery, automating discovery, that's something you really want, right? You want to push the system into you know areas beyond the training data. So you mentioned earlier or you asked me in the in the prep, how did we end up in in machine learning, right? In my group, and I mentioned we did it through category theory. And I actually when we s we did some work in 2017 on on neural networks, like surrogate models, and a thing that we sort of we we tested these things and the the stuff that really

you bothered me a lot is that yeah, these models are surrogates, but they're very boring actually, because they don't teach me anything new.

Philipp

Mm-hmm.

Markus J Buehler

they can make my my simulation faster. I I don't have to simulate final elements for four hours. I can do the whole thing in a minute. But I have to train it and it will not work outside the training distribution. So this is something that we that really irked me and and still does. And and now we've moved beyond this now with modern AI, but

That is something that I I was really always appalled by, actually. And and appalls maybe too strong of a word, but in the scientific process,

it's it's disappointing.

Philipp

Disappointing. Yeah.

Markus J Buehler

Yeah. And and so you really want to create machines that can tell that can teach you something new that is also real. It's not just imagine, hallucinate it, makes no sense. It's actually based

Philipp

Mm-hmm.

Markus J Buehler

on physics. It can be made. It's it's

Bruno Marnette

Hmm.

Markus J Buehler

something that is new, it's real, it can be explained, right? you can have a machine explain it to you. I I had a post

the other day, or actually today, and somebody asked me, so so if you're doing all this automation in science, how do you how do you actually how do you know how how how is it for you, how is it valuable for you? And I said, yeah, it's incredibly valuable because not only can I use this to discover new things, but the machine can even explain it to me. So so that was something that but you imagine running a big accelerator system, you get all the data, it doesn't explain the discovery to you. It might have made a new particle, you have no idea how it was made. This these machines we're building now

they can discover new things, but they can also talk to us. So I find that very, very interesting.

Philipp

Can I also suggest here something on top? Because you you make this connection between music and protein folding or protein creation. in mathematics, there is this very, very strong debate at the moment, what will mathematics look like? And, will we have theorems that we don't understand? But what you're suggesting is, first of all, you can ask obviously, but there is this kind of possibility even wilder that maybe to understand mathematics you will be listening to the music of it, right? And

Markus J Buehler

Yes.

Philipp

this this and it it could be the same for other fields that we are

crossing into the different medium in order to make it accessible for us humans.

Markus J Buehler

Yes.

Yes. absolutely. I totally agree with this. I think, yeah, I follow the debate obviously very closely in in math. And I mean I think in math is a sort of a different case. but for engineers and scientists, we have used machines that are better than us or can do things we can do for a long time. I mean, for us it's like every time I use a computer to run a billion atom MD simulation, it's not nothing I could ever do, right?

Philipp

Yes.

Markus J Buehler

but I mean the sort of the understanding of the system is definitely

you know reaching the human limit. I mean I want to say that in in some of the work we've done in recent months, the the sort of discovery or the the the behavior that the system can extract is far beyond sort of what humans can actually easily understand. I mean you you really have to work on it. And this is actually what I mentioned in my comment to this person on on on I think LinkedIn.

Is you have to do the intellectual work to actually go through and say, well, now I have this discovery. My machine tells me there's a principle here, or maybe a proof. And you really have to work with this machine and try to come through. And there might be a point where we cannot understand it. And I think, you know, we can maybe understand parts of it, and maybe like maybe we say maybe models of it, maybe it's it's a it's a translation into something else.

But but machines might help us with this for sure. And and it's also okay. we I think we have we're building now algorithms, machines that that are smarter than us. And that's I think we're right at the threshold. and there's incredible things we can do. But it's going to change our relationship with the world for sure. you know, f as a scientist, so for me, I have seen this. I am sometimes struggling with it as well. I mean, I tell

People, this is awesome. We're building these algorithms, but it means that much of the things I used to do is now done by machine. I because I when I teach my class, I'm teaching this modeling and simulation class right now. And I was just telling the students last week, well, if you know, two years ago, three, four years ago, right, you the only entity in the universe that could write the input file for this DFT model or this MD simulation or protein model was you, right? that was our job.

And the only person, the only entity that could analyze the data was also just you, right? It was us. That was our job. We had machines running the stupid, silly stuff, like doing the integration. Okay, but all the fancy intellectual work. Now that is no longer true. in fact, machines can can even design the the simulation instrument. They can write. I I've done this and I hadn't I have not published this yet, but we have done a study where we basically said, write your own molecular simulation code. You can look at other codes, but I want you to make your own.

And I want you to make

Philipp

Mm-hmm.

Markus J Buehler

sure it works by validating it. And and the the AI can do this actually. So it makes everything from scratch. It valids it, it does hundreds of different test cases. And it then runs it. It runs a p campaign of tests, experiments, and then it it assesses them, it looks at all the trajectories, and it writes a report to me. And this is really a big part of what scientists used to do. I in my field, material science. And machines are doing this now.

The question is what side are you gonna be on? are you gonna be pretending that this is not real? your life probably is not gonna be you're not gonna be very happy.

Philipp

Good luck, yeah.

Bruno Marnette

Mm-hmm.

Markus J Buehler

but if you are empowered, I mean I'm extremely empowered by this because I can do things now that I just couldn't do even just a few months ago. It's incredibly exciting. I I can I can try things, I can experiment with things, I can ask questions. For me, the learning and and doing this is really what I enjoy. but the level of insights that I can derive is so much higher than I

could do before. And that's great. as a scientist, that's what I enjoy doing. So I'm I'm to me, it's finding out about things about the world, learning. So the process, you you mentioned the the music exp the evolution of it in time. this process is exactly what discovery is it's it changes what I know about the world. That's what I enjoy doing. And that's what I write about in my papers. I we talk about this process. And building the machines to do this is just another level of abstraction. I think it will become very clear and maybe the next generation of scientists will be very native to them.

But there's definitely a big transition right now going on. It it it is changing the way we look and I can yeah, it's fundamentally changing what what it is to be a scientist for sure. Yeah.

Philipp

Yes.

Bruno Marnette

Talking about

changing the world, we've heard from you as a scientist, as a writer, and so on, but you're also an entrepreneur and you have a lab called Unreasonable Lab where I understand you're trying to deploy these creativity engines in the wild. I'd be very curious to hear your experience so far. What is it like to bring this technology to different companies, institutions, how are they using

Markus J Buehler

But

Bruno Marnette

it?

Markus J Buehler

yeah, I think a lot of industry is quite sort of I mean, think scientists are a little more in a in an ivory tower, you know, we're doing things more like if you an industry

Bruno Marnette

Yeah.

Markus J Buehler

and if you're an engineer, let's say, let's let's say you're an

Bruno Marnette

Yeah.

Markus J Buehler

engineer in industry, yeah, you you don't really care how you get the solution, right? So if you have a technology that can automate finding better materials and finding better solutions, yeah, you're gonna you're gonna take it. And especially your competitors are gonna take it. So that's my experience. I think that's huge interest in industry.

for these technologies and industry is not as dogmatic maybe in in how do we what's pure science, what's pure math, they are driven by by by solving problems. And this is the the thing that you see actually a lot of in the in the entrepreneurship world of course and and in this in a sense is you you you have many more problems than we have solutions for. and and that's also the I the argument of course we

Philipp

Ha

Markus J Buehler

always make when people say, well this is gonna take jobs away. Well no, because

I have many more things I want to solve as a scientist, as a material scientist, I want to invent many, many new materials that I just in

Bruno Marnette

Mm-hmm.

Markus J Buehler

the older way of doing things that would have taken me many lifetimes, right? Many universe times actually.

Philipp

Yeah.

Markus J Buehler

if I can accelerate this and I can maybe have AI and quantum computing and I can simulate many different worlds of possibilities, I can do it more quickly. So the experience is that I think in industry there's a really strong appetite for this. There's also a very strong appetite in in industry, of course. they understand

in creating commercial products, that the the value of the data they have. So something that we we see

Bruno Marnette

Mm-hmm.

Markus J Buehler

a lot working with, you know, industry customers and so on, is a lot of times you know, they're a lot more of course, you know, concerned about sharing data with you know, a model provider. Right. So so what are the

Bruno Marnette

Mmm.

Markus J Buehler

the so there's a lot of engineering in fact required to sort of make give confidence to to these kind of and in you know

customers and clients. but yeah, generally it's a revolution. I think there probably is a sort of a very practical thing is that you you work in industries that have huge potential for AI applications. they have the least AI talent. So right so

Philipp

Mm-hmm.

Markus J Buehler

if you're

Bruno Marnette

Mmm.

Markus J Buehler

imagine this you know mid-sized company maybe in Europe, Germany has a lot of these mid Middlestand, right? And so for example there, I mean there's an enormous data, enormous insight, know-how these companies

They have huge problems, they can really have huge impact on AI because they've never automated anything really, potentially,

right?

Bruno Marnette

Mm-hmm.

Markus J Buehler

So but they don't have the talent. So making this connection is something I found there's a lot more possibilities and work or projects and clients than you can possibly do. And that's maybe the one thing that I I learned is of course also that that to automate this process is not as easy as it sounds, perhaps. There's this there's a lot of engineering, right?

Philipp

Yeah, one hundred percent.

Markus J Buehler

Required to

To connect that data from that company to that problem and with the constraints of we wanna have sandboxed everything, air gapped everything, because our data is our IP and right. And so so there's a lot of real world engineering that I think is just at play. So when you're looking at the frontier of AI, probably you know, we could be doing a lot of engineering for many, many years, even with today's technologies.

Philipp

Yeah.

Markus J Buehler

to to make a big impact. So it's it's an exciting time. and I think yeah time is probably the only thing that we don't have enough of because

Philipp

Yeah.

Markus J Buehler

we we need more time, more people, more talent, yeah, to to do all

Bruno Marnette

in.

Philipp

Yeah. Markus we took already a lot of your time. Maybe closing on one thing. So you kind of in by in passing in bypassing you mentioned when you look back in your your how you trace your own intellectual kind of development over time and the questions that you were working on. it looks quite different from the from from looking back from as from looking at the beginning, obviously, as what you imagined.

Now this is a hard question, but what will you be working on next and how do you think? Now, with knowledge that again, you look back differently, what will you think? What happens next to you? What will you be working on? What's your the main thing that you want to focus on in the let's say next two years?

Markus J Buehler

Yeah. Well, I think I've learned actually, so I learned a lot about what I have done and why I've done things retrospectively. So you look back, right?

Philipp

Mm-hmm.

Markus J Buehler

And so one of the things I I mentioned is is sort of, you know, what I've been trying to do is to find these algorithms that explain many different things, to think taking things apart, rebuilding them and kind of figuring that out. And and that's something actually that is pretty constant in my in my career. I've also been really interested in breaking things. So I've always

Bruno Marnette

Mm-hmm.

Markus J Buehler

I mean I worked on fracture, I worked in fracture of biological materials.

broke and now I'm breaking world models. So that's the theme. And it's really is truly true actually. So I think there are intellectual through lines that I see in my own work that some of these are not designed. It's not like I woke up in the morning and I said I wanna but I I realized that the way as societies you formed and and what you're doing when you're young, you know, and you're building, you know like for example, I I was cu very curious about chemistry and mathematics and and how things sort of complex things emerge. That's a through line that came that you would have

Talk to me maybe when I was an undergrad student, I was very interested in that, and I'm still very interested in that. Now I'm building models of intelligence. So that's sort of how I evolve this. I mean, the biggest thing right now for us probably is the sort of next direction is building machines that build other models and other machines. That that is really something that we have, I mean, always wanted, guess, as a species and and as maybe early computer science, early AI, of course. this is actually something also I've been doing a lot recently reading.

Very old AI papers, right? Like I mean up like the old Dartmouth you know, this initiative that they had or the conference there,

Philipp

Workshop, yeah.

Markus J Buehler

right? So it actually is very interesting because you see a lot of the stuff that people have talked about back then, even before that, of course. I mean, found a paper for one of the talks in France from 1700. I think Dumatre was the name. I don't know if you've seen of that, but it's basically this guy in France, or I think it was France or Netherlands. Basically he said, the the mind is a machine, he said. And

It's basically some sort of mechanical thing. And he wasn't talking about philosophy, he was talking about the actual intelligence. And he said, Well, this is machine. We can build it. Okay. It was sort of totally cast aside from society. I think he was jailed and maybe killed. I don't know if was killed, but sort of killed, at least intellectually. so so we've always actually had sort of this idea, and I think now we're able to do this. We're actually building machines, and these machines can build other machines.

And it's an incredible exciting word. So for me, yeah, it's about building machines that can make discoveries. And I wanna be part of this process. I also have no problem in being amazed by by that. I mean, I'm always amazed by nature. If you study biology, right? Yeah. So that's another thing different from mathematics. I mean, biology is something that we study and we have no idea how it works, even the most basic stuff.

Philipp

Yeah.

Markus J Buehler

Right? and so for me studying biological organism, I'm in wonder and it's something that's beyond my cognition. I don't

I I cannot explain how this exactly works. I have glimpses of stuff, but and it's very humbling. So I think in AI and in sort of this world we're gonna be like this too. We're gonna see maybe a machine that can explain how a cell works. Maybe it it models it and it can actually design synthetic cells, synthetic life. And we might not under be able to understand it, but the machine can h try to help us sort of get through there, right? but if I can do this, or I can find new energy sources, fusion, I can make new materials that

Allow us to contain, these systems be made incredible. Maybe it's a new compute paradigm. So I I look into the world of really creating algorithms that make other algorithms. That's been something I'm very interested in. I also think that it's going to change. you know, I I look at this also, I've been an educator, of course, as well, and I I so I'm I have all these students in front of me all the time. And they come in and they say, what do I do with my time? I'm I'm

Philipp

Yeah.

Markus J Buehler

here now, I'm I'm I'm getting a PhD

Bruno Marnette

Mm-hmm.

Markus J Buehler

at MIT.

What am I getting it for? And so we're gonna have to answer that question. And I you know, I think we wanna train people that have are empowered by these very powerful tools and that we make them part of the process. we wanna be able to control them, we want to make sure they're ethical, they they help the world, they help us. but I'm very you know, I'm very positive about accelerating this process, and I think that this is in fact the only way we will go forward. I mean, history tells us that I mean, I think in we really can't

Hold up this development. It's been tried before. is it right? So, in

Bruno Marnette

Mm.

Markus J Buehler

in Germany, and I grew up there, right? So the the printing press is a perfect example. It was try to make it try be, it was something that empowered a huge swath of people to learn and read by themselves and to be able to have access to information. And and that was sort of something that was very threatening to the the feudal or feudal powers at the time.

And I think that something like this is happening again with AI. And and I actually do think that that there will be probably much bigger fights ahead of us than we currently see.

Philipp

yeah.

Markus J Buehler

Because there's gonna be it's gonna change fundamentally who are the who are the institutions in power, the the classes in power, and and there's gonna be new kinds of groups and classes or whatever you call them are gonna become much more powerful than they were before.

And that's gonna create a a big conflict potentially. So we hope to navigate this. And that's something at MIT we're really trying to do. We're trying to be talk about the ethics of this and and what it means and how do we empower the world. our mission at MIT is to make the world better, bring technology

Bruno Marnette

Mm-hmm.

Markus J Buehler

to the world and and science and and help. And and this is just another iteration of this. But it's not it's very hard intellectually because it it like we said earlier, it it is the first technology that is essentially

Challenges us and maybe is smarter than us, definitely in certain areas.

Philipp

Yeah. Yeah, I think also the the timelines in which these things are evolving are so fast

Markus J Buehler

Yes.

Philipp

that normally our institutions are used to a very

Markus J Buehler

Right.

Philipp

different timescale to adapt basically.

Markus J Buehler

Yes, it's impossible. Yeah. And universities have particularly so that's you mentioned about s the startup world. I mean the the speed there is much faster. You can move more quickly.

Bruno Marnette

Mm.

Markus J Buehler

Universities obviously, yeah, they and there's a good reason for this. I mean they they say, I mean, we say of course, yeah, it's important that we have the sort of more, you know, reflective attitude. But yeah, you're totally right. we it's it's it's very difficult. you know, the students are a lot faster and and some people like myself, you know, we work at the frontier, we know we use the tools, but there are many

in our community in science and I came back from some Gordon conferences in the summer. Incredible work, but a lot of you know, there's a lot of people that have no AI use at all. And I could sort of walk around and tell them, hey, you could really accelerate things here, you could do things there. and and I think it creates a sort of a divide. And there's also and there's honestly there's a lot of pushback actually in in in some, you know, some of the young people, I mean some of the students coming in, they hate AI. They

They want nothing to do with

Bruno Marnette

Mm.

Markus J Buehler

it. Some of them for environmental reasons, some of them because they just they just don't like it, they're threatened by it. And and this happens also in science. I mean, there are people, groups that are really sort of opposed to this. yeah, I think I think the the speed has definitely something to do with it. It it takes it it takes an enormous engagement that actually to actually be be part of this. If you do if you just observe it and read about it in the news, yeah, you're gonna be I mean, I I sometimes ask people, did you hear about this new like on the weekend, I said, Did you hear about this new

discussion we had with anthropic and everything, and we were like, what last week? No, it's this Saturday or Sunday, right? it's so quick. The debate is so is evolving so quickly that it's like, yeah, you have to fully engage with it. You can't be an observer on the sideline. And and most people of course are just absorbing it. And and it it it it seems very overwhelming in that in that way, for sure. I mean it's it's very difficult. And I think this is something we also we all have to do a really good job, a better job probably

in in bringing people along in this in this journey with the speed of of change.

Philipp

Yes.

Bruno Marnette

That's something I'm

very impressed by in your career and in what you're saying is you seem to be talking about the creative side of AI with the same kind of ambition and optimism as the what I would call the intelligence side, right? And I find it actually quite rare because you have many leading figures in AI. I'm thinking Demis Hassabis for instance, who obviously someone is extremely creative as a person and he speaks a lot about the beauty of

the arts and is a very pro-multidisciplinary research person. But if you see what he's able to do, say, at Google, the focus has really been more on verifiable reward, reinforcement learning, winning games, domains where winning is kind of clear. And I think at the same time, also, these companies try to scaring people into saying, we're to do your creative work. They're still trying to.

draw a line between, with just a calculator, you still, you do the taste thing. But you're one of the, I guess, a few people we've found so far who really feel comfortable working on both sides. I wonder like how much, I guess how much pushback you get. Like are people sometimes a bit upset to see you trying to automate creativity in a way? do you get...

Markus J Buehler

I I for sure, yeah. I I give talks, I get questions sometimes about this. And I've given actually I

Bruno Marnette

You

Markus J Buehler

think it was a talk at NeurIPS last year, so I give a keynote there and and people were saying, Well, I think somebody came up to me, I th I remember it was doing the t after the talk after after the talk to me personally, but they basically said something to that effect and and they're like, you know, are taking basically you're you're you're you're taking our jobs away. What are you doing, right? And unless well and you know, give

Bruno Marnette

Yeah.

Markus J Buehler

the insight I gave earlier, but no, I think I think actually

To me, these open-ended problems are the most interesting ones because the things you can measure, they're easier. you can s design a reward and you can find a model.

Bruno Marnette

Exactly.

Markus J Buehler

The the really interesting thing is actually going to spaces we have not been before. And and this is actually it touches upon all my work. And actually being a scientist, being an engineer, this is the space we're usually in. I mean, I'm if I'm solving a problem, like if I'm okay, there are different types of scientists, of course, but if I'm trying to develop a better I don't know if you've been with force fields, like like these are models of molecules.

So if you're trying if your job becomes I want to develop a more accurate force field than my colleagues have done, this is a totally fine career you can have, but you're not gonna develop something. You're gonna spend your time making something that others have already defined, right? And you're just optimizing in this. But you can also ask the question, is there a totally different way of doing it? maybe I'm not I'm

Bruno Marnette

Mm.

Markus J Buehler

developing a totally different paradigm. And I've always been drawn to these more open-ended problems. And yeah, I think I think you get a certain, I mean I think in the AI world for sure for automation.

I've gotten questions, I've gotten comments from people colleagues at my T ask the question. not so much personally to me, but just in general, they're saying, well,

Bruno Marnette

Mm.

Markus J Buehler

what is my role going to be? I'm

Bruno Marnette

Yeah.

Markus J Buehler

a lot of these institutions are based on the idea that you have elite knowledge, we have elite networks that only I can unlock. So so

Philipp

Mm-hmm.

Markus J Buehler

if you come

Bruno Marnette

I see.

Markus J Buehler

here as a student, right? But of course with AI, that's out of the door. if a student

Bruno Marnette

Yeah

Markus J Buehler

anyone in the world can basically

access the the all the nuances of knowledge. Maybe if not today, then for sure in the future. So the role of these elite institutions gonna change and a lot of identities based on this. Right. So yeah, this doesn't answer the question about the the creativity, but I I have no problem with this. And actually I w I wanna say that so in the art community, people in art actually are quite interested in the machine creativity and they're actually quite interested in using it. Right. So I've done of course this is a self selected group that I'm working with because I've worked with people maybe but

Bruno Marnette

Yeah.

Markus J Buehler

But I have had I've had some really great experiences. there are people that are interested in saying, Well there's a new tool. So I I can I can

Philipp

Yeah.

Markus J Buehler

now access things that I couldn't do before. It's incredible.

Bruno Marnette

Yeah.

Philipp

artists undergone this before, right? oil was invented for paintings, then the photograph came, then electronic

Markus J Buehler

Right.

Philipp

music came. So these kind of distortions or radical changes, not distortions, radical changes

Markus J Buehler

Yes. Yes.

Philipp

have been there.

Markus J Buehler

Yes. exactly.

Bruno Marnette

Yeah.

Markus J Buehler

And and actually, and this is a great point. And and in fact, the the more radical these transitions are, we talked about earlier, the more dis more important the discoveries are, of course. If you if you're scratching on the on the known edges, that's boring. If you're transforming things like I mean, you know, you know, sort of coming up with a totally new paradigm, asking a totally new question, or saying, Well the human I very provocative in art we don't need humans anymore. We just need machines and humans observe.

Maybe that that is actually a form of art, actually. And so that's the

Philipp

Yeah.

Bruno Marnette

Yeah.

Markus J Buehler

flexibility you have in art. you can actually go and and define a world, that's what I mentioned earlier, that is not based in physics or and maybe it's based on an imagination. And actually this exact exercise gives you access to a new latent representation of something that you might then unlock in science. Like so creating a a world in which maybe no art is created by humans and we're only observing it. That is a is a

maybe a beautiful art experiment actually to do. So it is there are different schools of thought, of course, for sure. I think in in music I've seen actually a lot in music performance, this is sort of like the math, very difficult, you know, if you're also in music composition, I mean, you know, can use I mean, if you are i writing music is extremely hard. you know, the the whole process, the whole craft of of writing it, and act performing it and recording it and other things.

Now if you you can use AI tools to do a much better job than 99.9% of humans. In fact, producing music actually I mean producing it and making it sound nice at the end, mixing it is a process that's an art form or like an engineering slash art. Machines are actually already much better than that for most of the average purposes.

Bruno Marnette

Hmm.

Markus J Buehler

So it's very humiliating to see this. But I again I'm pretty I I use it as a tool.

Okay, now I have a tool. I can I can use the machine to help me do things I couldn't do before. But yeah, I mean there's definitely different different attitudes for sure.

Philipp

Yes. maybe also doubling down on what Bruno said, I think it from my side as well, I th I find it just very refreshing that you're so much focused also on the positivity and also on the empowerment, right? you talk about students and indeed a student can be stuck. I my job is to write papers, but a student can also make this transition transition into my job is building a machine because this machine will replicate and will be part of it. And maybe if I'm the inventing a biological machine, an economist can pull it in and reform.

economics, etcetera. I think this is just it's awesome. And it thanks again for being part of this. It it was really awesome.

Markus J Buehler

Yes. Yeah, exactly. Yeah, and and you're right.

about the the machine, you know, like in biology, in biological engineering, that's what we've been doing for a long time. We've been trying to build tissues and we're building these machines, we have no idea how they work. you know, we we make,

Bruno Marnette

Yeah.

Markus J Buehler

you know, some new tissue with stem cells and doing some biology, some some chemistry on it, and you know, kind of it's all there's some theories, but it's really hand waving, right? And

Philipp

Yeah. Yes.

Markus J Buehler

Yeah, so exactly. So a student does this, they we're trying to kind of reverse engineer what does this machine do? How does it work? It's it it works, but how exactly? We've no idea. so I think it's very common in in engineering. Yeah, yeah, exactly. Yeah. But as yeah, I mean I think the the students what I tell my students always is they they have to understand it, especially if they're getting a PhD for it, right? So that's a that's a question of of credit or when you write a paper. You can have a machine do it, but if you're an author and you're presenting it, you have to

You have to be the narrator of whatever exercise you went through. And and you have to, yeah, use it as a tool. Learn something that I didn't know before. And that's the message of the paper. Like that is something that I see a little bit as different, right? So if you're coming in as a student and you look at this and say, I don't have to do anything. I can just when I'm meeting with my yeah, yeah,

yeah, yeah, just make make make slides with Claude and that's it. And that is something that I tell students, you you wanna be, you wanna be part of the process. Otherwise, there's no point. I mean.

So you don't have to get a PhD. So that's the one rule I have. But they

Bruno Marnette

Yeah.

Markus J Buehler

can use it for sure and they should use it. and they all are using it now. yeah, I had a I had a discussion.

Philipp

Yes.

Markus J Buehler

five years ago, four years ago, so the we built the early agenda system back then. And I remember I we presented them in a group meeting and I had one student raise her hand. I'm not gonna say who it was, but she said, This is pretty interesting, it's also really scary. It's gonna take my j it's basically what I'm doing. And I said,

Philipp

Yeah.

Bruno Marnette

Yeah.

Markus J Buehler

Yeah. And and back then it was sort of really remote. And now I mean

Times have changed in the sense that everyone's yeah,

Philipp

It's obvious. Yeah.

Markus J Buehler

it's very obvious and everyone's using it. And if you're not using it, you're you're really behind. So yeah.

Bruno Marnette

Thank

Philipp

Yeah, and that's the other thing.

Markus, thanks a lot for being part of it. It was really, really nice.

Bruno Marnette

Thanks, Silvestre.

Markus J Buehler

Yeah, thank you so much. It was a great

conversation. I enjoyed it and yeah, thanks so much