0:00

The following is a conversation with George Hotz,

0:03

his third time on this podcast. He's

0:05

the founder of Kama.ai that

0:07

seeks to solve autonomous driving and

0:09

is the founder of a new company called TinyCorp

0:13

that created TinyGrad, a neural

0:15

network framework that is extremely simple

0:18

with the goal of making it run on any

0:20

device by any human

0:23

easily and efficiently. As

0:25

you know, George also did a large number

0:27

of fun and amazing things from hacking

0:29

the iPhone to recently joining Twitter

0:32

for a bit as a

0:34

intern in quotes, making the case

0:36

for refactoring the Twitter code base.

0:39

In general, he's a fascinating engineer

0:41

and human being and one of my favorite people

0:43

to talk to.

0:46

And now a quick few second mention of each sponsor.

0:48

Check them out in the description. It's the best way

0:50

to support this podcast. We've got Numerai

0:53

for the world's hardest data science tournament,

0:56

Babbel for learning new languages, NetSuite

0:59

for business management software, InsightTracker

1:01

for blood paneling, and AgeeOne for

1:03

my daily multivitamin. Choose

1:05

wisely, my friends.

1:07

Also, if you want to work on our team, we're

1:09

always hiring, go to lexfriedman.com

1:11

slash hiring. And now onto

1:13

the full ad reads. As always, no ads in the middle.

1:16

I try to make this interesting, but if you must

1:18

skip them, friends, please still check out our sponsors.

1:20

I enjoy their stuff. Maybe you will too.

1:24

This episode is brought to you by Numerai,

1:26

a hedge fund that uses artificial intelligence

1:29

and machine learning to make investment decisions. They

1:31

created a tournament that challenges data

1:33

scientists to build

1:35

best predictive models for financial markets. It's

1:37

basically just a really, really difficult

1:40

real world data set to test out

1:43

your ideas for how to build machine learning

1:45

models. I think this is a

1:47

great educational platform. I think this is a great

1:49

way to explore, to learn about machine

1:51

learning, to really

1:53

test yourself on real world data

1:55

with consequences. No

1:57

financial background is needed. The models are

1:59

scored.

1:59

based on how well they perform on unseen

2:02

data. And the top performers receive a share

2:04

of the tournament's prize pool.

2:06

Head over to numeri.ai to sign up for a tournament

2:13

and hone your machine learning skills. That's numeri.ai

2:16

for a chance to play against me and win

2:19

a share of the tournament's prize pool.

2:22

That's numeri.ai. This

2:24

show is also brought to you by Babbel, an

2:26

app and website that gets you speaking

2:29

in a new language within weeks. I

2:31

have been using it to learn a few languages, Spanish,

2:35

to review Russian, to practice Russian, to

2:37

revisit Russian from a different

2:39

perspective because that becomes more and

2:42

more relevant for some of the previous

2:44

conversations I've had and some upcoming conversations

2:46

I have. It really is fascinating

2:48

how much another language,

2:51

knowing another language, even to

2:53

a degree where you can just have little

2:55

bits and pieces of a conversation, can

2:57

really unlock an experience in another part of

2:59

the world. When you travel in France

3:02

and Paris, just having a few

3:04

words at your disposal, a few phrases, it

3:06

begins to really open you up

3:09

to strange, fascinating new

3:11

experiences that ultimately,

3:13

at least to me, teach me that we're all the same.

3:16

We have to first see our differences to

3:18

realize those differences are grounded in

3:20

a basic humanity. And that

3:23

experience that we're all very different

3:25

and yet at the core the same.

3:27

I think travel with the aid

3:29

of language really helps

3:31

unlock. You

3:35

can get 55% off your Babbel subscription

3:38

at babbel.com slash LexPod. That's

3:40

spelled B-A-B-B-E-L.com

3:43

slash LexPod. Rules and restrictions

3:46

apply.

3:47

This show is also brought to you by NetSuite,

3:51

an all-in-one cloud business management system.

3:53

They manage all the messy stuff

3:56

that is required to run a business,

3:58

the financials, the human resources.

3:59

to the inventory if you do that kind of thing, e-commerce,

4:03

all that stuff, all the business-related details.

4:05

I know how stressed I am about

4:08

everything that's required to

4:10

run a team,

4:12

to

4:13

run a business that involves

4:16

much more than just ideas and

4:18

designs and engineering. It involves

4:20

all the management of human beings, all

4:22

the complexities of that, the financials,

4:25

all of it, and so you should be using the best

4:27

tools for the job.

4:29

I sometimes wonder if

4:31

I have it in me, mentally

4:34

and

4:35

skill-wise,

4:37

to be a part of

4:39

running a large company.

4:42

I think like with a lot of things in life, it's

4:44

one of those things you shouldn't wonder too much about.

4:47

You should either do or not do.

4:50

But again,

4:52

using the best tools for the job is

4:54

required here. You can start now

4:56

with a no payment or interest for

4:58

six months. Go to netsuite.com slash

5:00

Lex to access their one of a kind financing

5:03

program. That's netsuite.com

5:06

slash Lex. This

5:08

show is also brought to you by Insight Tracker,

5:11

a service I use to track biological data,

5:14

data that comes from my body, to predict,

5:16

to

5:17

tell me what I should do with my lifestyle,

5:19

with my diet, what's working and what's

5:21

not working. It's obvious,

5:25

all the exciting breakthroughs that are happening with

5:27

transformers, with large language models,

5:30

even with diffusion, all of

5:32

that is obvious that with raw

5:34

data, with huge amounts of raw data,

5:36

fine tuned to the individual, would

5:38

really reveal to us

5:41

the signal in all the noise of

5:43

biology. I feel like that's on the horizon.

5:47

The kinds of leaps in development

5:49

that we saw in language

5:51

and now more and more visual data.

5:53

I feel like biological data is around the corner,

5:56

unlocking what's there in this

5:58

multi-hierarchical.

5:59

distributed system that is our biology.

6:02

What is it telling us? What is the secrets it holds?

6:05

What is the thing that it's

6:07

missing that could be aided?

6:09

Simple lifestyle changes, simple diet changes,

6:12

simple changes in all kinds of things that are controllable

6:14

by individual human being. I can't wait

6:16

till that's a possibility and InsideTracker

6:18

is taking steps towards that. Got special savings

6:21

for a limited time when you go to insidetracker.com

6:23

slash Lex.

6:26

This show is also brought to you by Athletic

6:28

Greens. That's now called AG1. It

6:31

has the AG1 drink. I drink it twice

6:33

a day. At the very least, it's an all-in-one

6:35

daily drink to support better health and peak performance.

6:38

I drink it cold, it's refreshing,

6:40

it's grounding. It helps me

6:44

reconnect with the basics,

6:47

the nutritional basics that makes this whole machine

6:49

that is our human body run. All

6:52

the crazy mental stuff I do for work,

6:56

the physical challenges, everything,

6:59

the highs and lows of life itself.

7:02

All of that is somehow made better knowing

7:04

that at least you got your nutrition

7:06

in check. At least you're getting enough

7:08

sleep. At least you're doing the basics.

7:11

At least you're doing the exercise. Once

7:13

you get those basics in place, I think

7:15

you can do some quite

7:17

difficult things in life.

7:19

But anyway, beyond all that is just a source

7:21

of happiness and a kind of

7:23

a feeling of home.

7:26

The feeling that comes from returning

7:28

to the habit time and time

7:30

again. Anyway, they'll give

7:32

you one month supply of fish oil when you sign

7:34

up at drinkag1.com

7:37

slash Lex.

7:39

This is the Lex Revenant podcast. To

7:42

support it, please check out our sponsors in the description.

7:45

And now, dear friends, here's George

7:47

Hotz.

7:49

He is not there anymore, so he can pass figures from the

7:52

sources. This unreal account has made it lessstallable

7:56

compared to colonialism. We have only three

7:58

who have grave You

8:05

mentioned something in a stream about

8:07

the philosophical nature of time. So

8:10

let's start with the wild question. Do you think

8:12

time is an illusion?

8:14

You know, I

8:17

sell phone calls to Kama

8:19

for $1,000. And some guy called

8:21

me and like,

8:24

you know, it's $1,000. You can talk to me for half an hour.

8:27

And he's like, yeah, okay. So

8:29

like,

8:30

time doesn't exist. And I really wanted

8:32

to share this with you. I'm like,

8:34

oh, what do you mean time doesn't exist? Right?

8:37

Like, I think time is a useful model, whether it

8:39

exists or not, right? Like does quantum

8:41

physics exist? Well, it doesn't matter. It's

8:44

about whether it's a useful model to describe

8:46

reality. Is time

8:49

maybe compressive? Do

8:51

you think there is an objective reality or is

8:53

everything just useful models? Like

8:56

underneath it all. Is there an actual

8:59

thing that we're constructing models for?

9:03

I don't know. I was

9:05

hoping you would know. I don't think it matters. I

9:07

mean, this kind of connects to the models

9:10

of constructive reality with machine learning. Right?

9:13

Sure. Like, is it

9:15

just nice to have useful approximations

9:18

of the world such that we can do something with it? So

9:21

there are things that are real. Column

9:23

of graph complexity is real. Yeah.

9:25

Yeah. The compressive thing. Math is real.

9:28

Yeah. This should be a t-shirt. And

9:31

I think hard things are actually hard. I don't think P

9:33

equals NP. Ooh, strong words.

9:36

Well, I think that's the majority. I do think factoring

9:38

is in P, but. I don't think you're the

9:40

person that falls the majority in all walks of

9:43

life, so. Well, good help for that one I do. Yeah.

9:46

In theoretical computer science, you're one of the sheep. All

9:49

right. But to you,

9:52

time is a useful model. Sure.

9:54

What were you talking about on the stream?

9:57

What time? Are you made of time? I

9:59

remembered half the thing. I said on stream. Ah.

10:02

Someday someone's going to make a model of all of it and

10:04

it's going to come back to haunt me. Someday soon?

10:07

Yeah, probably. Would that be

10:09

exciting to you or sad that there's

10:11

a George Hotz model?

10:13

I mean the question is when the George Hotz model

10:16

is better than George Hotz.

10:18

Like I am declining and the model is growing.

10:20

What is the metric by which you measure better or worse

10:23

in that if you're competing with yourself?

10:25

Maybe you can just play a game

10:28

where you have the George Hotz answer and the George Hotz model

10:30

answer and ask which people prefer. People

10:32

close to you or strangers? Either

10:35

one. It will hurt more when it's people close to me but

10:38

both will be overtaken by the George

10:40

Hotz model. It'd

10:42

be quite painful, right? Loved ones, family

10:45

members

10:46

would rather have the model over for Thanksgiving

10:48

than you. Yeah.

10:51

Or like significant others would

10:53

rather sext. Like

10:58

with the large language model version of you.

11:00

Especially when it's fine tuned

11:02

to their preferences. Yeah.

11:06

Well, that's what we're doing in a relationship, right?

11:08

We're just fine tuning ourselves but we're inefficient with

11:10

it because we're selfish ingredients and so on.

11:13

All language models can fine

11:15

tune more efficiently, more selflessly. There's

11:17

a Star Trek Voyager episode where, you know, Catherine

11:20

Janeway, lost in the Delta Quadrant

11:22

makes herself a

11:24

lover on the holodeck. And

11:28

the lover falls asleep on her arm and he

11:30

snores a little bit and, you know, Janeway edits the

11:32

program to remove that. And then

11:34

of course the realization is, wait,

11:37

this person's terrible. It is actually all their

11:41

nuances and quirks and slight annoyances that

11:43

make this relationship worthwhile. But

11:46

I don't think we're going to realize that until it's too late.

11:48

Well, I think

11:51

a large language model could incorporate the

11:54

flaws and the quirks and all that kind of stuff. Just

11:56

the perfect amount of quirks and

11:59

flaws to make you...

11:59

charming without crossing the line. Yeah,

12:02

yeah. And that's probably a good

12:06

approximation of the, the percent

12:08

of time the language model

12:10

should be cranky or

12:14

an asshole or jealous

12:16

or all this kind of stuff. And of course it can and it will,

12:19

but all that difficulty at that point is artificial.

12:22

There's no more real difficulty. Okay,

12:24

what's the difference between real and artificial? Artificial

12:27

difficulty is difficulty that's like constructed

12:30

or could be turned off with a knob. Real

12:32

difficulty is like you're in the woods and you

12:34

gotta survive.

12:36

So if something can not

12:38

be turned off with a knob, it's real.

12:42

Yeah, I think so. Or I mean, you

12:44

can't get out of this by smashing the knob with a hammer.

12:47

I mean, maybe you kind of can, you know, into

12:50

the wild when, you know,

12:53

Alexander Supertramp, he wants to explore

12:55

something that's never been explored before, but it's

12:57

the 90s. Everything's been explored. So he's like, well, I'm

13:00

just not gonna bring a map. Yeah.

13:02

I mean, no,

13:04

you're not exploring. You should have brought

13:06

a map, dude, you died. There was a bridge a mile from where you were camping.

13:09

How does that connect to the metaphor of the knob? By

13:13

not bringing the map, you didn't

13:15

become an explorer.

13:17

You just smashed the thing. Yeah.

13:19

Yeah. The difficulty is still artificial.

13:22

You failed before you started. What if we just don't

13:24

have access to the knob? Well, that

13:27

maybe is even scarier, right?

13:29

Like we already exist in a world of nature and nature

13:31

has been fine-tuned over billions of years to

13:36

have humans build

13:39

something

13:41

and then throw the knob away in some grand romantic

13:44

gesture is horrifying.

13:46

Do you think of us humans as individuals

13:48

that are like born and die, or is

13:51

it, are we just all part of one

13:53

living organism that is

13:56

earth, that is nature? I

13:59

don't think there's a clear. line there, I think

14:01

it's all kind of just fuzzy. I don't know.

14:03

I mean, I don't think I'm conscious. I don't think

14:05

I'm anything. I think I'm just a computer

14:08

program.

14:09

So it's all computation. I think running

14:12

in your head is just a computation.

14:15

Everything running in the universe is computation, I think.

14:17

I believe the extended church starting thesis.

14:20

Yeah, but there seems

14:22

to be an embodiment to your particular computation.

14:24

Like there's a consistency.

14:26

Well, yeah, but I mean, models have consistency

14:28

too. Models

14:31

that have been RLHF'd will continually

14:33

say, you know, like, well, how do I

14:36

murder ethnic minorities? Oh, well, I can't

14:38

let you do that, Al. There's a consistency to that behavior.

14:40

So RLHF,

14:43

like we all RLHF each other. We

14:48

provide human feedback and thereby fine-tune

14:51

these little pockets

14:53

of computation. But it's still unclear why that

14:56

pocket of computation stays with you like

14:58

for years. It just kind of follows like

15:01

you have this consistent

15:04

set of physics biology.

15:06

What like

15:10

whatever you call the neurons firing,

15:12

like the electrical signals, the mechanical signals, all

15:14

of that that seems to stay there and it contains

15:16

information, stores information and

15:18

that information permeates through time

15:21

and

15:22

stays with you. There's like memory, there's

15:25

like sticky. Okay, to be fair,

15:27

like a lot of the models we're building today are

15:29

very, even RLHF is nowhere

15:32

near as complex as the human loss function. Reinforcement

15:34

learning with human feedback. You

15:37

know, when I talked about will GPT-12 be

15:39

AGI, my answer is no, of course not. I mean,

15:42

cross-entropy loss is never going to get you there. You

15:44

need probably

15:46

RL in

15:47

fancy environments in order to get something that

15:50

would be considered AGI-like. So

15:53

to ask the question about why, I don't

15:56

know, it's just some quirk of evolution.

15:58

I don't think there's anything particularly...

15:59

special about

16:02

where I ended up, where humans

16:04

ended up.

16:06

So, okay. We have human level

16:08

intelligence. Would you call that AGI?

16:11

Whatever we have, G-I? Look,

16:13

actually I don't really even like the word AGI,

16:16

but general intelligence is

16:18

defined to be whatever humans have.

16:20

Okay. So why can GPT-12

16:23

not get us to AGI? Can we just like,

16:26

link on that?

16:27

If your loss function is categorical cross entropy,

16:30

if your loss function is just try to maximize

16:32

compression, I have a sound

16:34

cloud, I rap, and I tried to get chat

16:36

GPT to help me write raps. And

16:39

the raps that it wrote sounded like YouTube common

16:41

raps. You know, you can go on any rap beat online and

16:44

you can see what people put in the comments. And it's the most

16:46

like

16:47

mid quality rap you can find. Is

16:49

mid good or bad? Mid is bad. Mid is bad.

16:52

Like mid, it's like. Every time I talk

16:54

to you, I learn new words. Mid.

16:58

Mid, yeah.

16:59

I was like, is it like basic?

17:01

Is that what mid means? Kind of, it's like

17:04

middle of the curve, right? So there's

17:06

like, there's like that intelligence curve.

17:09

And you have like the dumb guy, the smart guy, and then

17:11

the mid guy. Actually being the mid guy is the worst. The

17:14

smart guy is like, I put all my money in Bitcoin. The mid

17:16

guy is like, you

17:16

can't put money in Bitcoin. It's not real money.

17:21

And all of it is a genius meme. That's

17:24

another interesting one.

17:25

Memes. The humor,

17:28

the idea, the absurdity, encapsulated

17:31

in a single image. And it

17:33

just kind of propagates virally

17:37

between all of our brains. I

17:39

didn't get much sleep last night. So I'm very, I

17:41

sound like I'm high, but I swear I'm not. Do

17:45

you think we have ideas or ideas have

17:47

us?

17:49

I think that we're going to get super scary

17:52

memes once the AIs actually are

17:54

superhuman. Do you think AI

17:56

will generate memes? Of course. Do

17:58

you think it'll make humans laugh? I

18:00

think it's worse than that. Infinite

18:04

jest, it's introduced in the first 50

18:06

pages, is about a tape that

18:09

once you watch it once, you only

18:11

ever want to watch that tape.

18:12

In fact, you want to watch the tape so much that someone

18:15

says, okay, here's a hacksaw, cut off your

18:17

pinky, and then I'll let you watch the tape again, and

18:19

you'll do it. We're

18:21

actually going to build that, I think, but it's not going

18:23

to be one static tape. I think the human brain

18:25

is too complex

18:27

to be stuck in

18:29

one static tape like that. If you look at ant

18:31

brains, maybe they can be stuck on a static tape. We're

18:35

going to build that using generative models. We're

18:37

going to build the TikTok that you actually can't look

18:39

away from.

18:41

TikTok is already pretty close there, but the

18:43

generation is done by humans. The

18:45

algorithm is just doing their recommendation, but if

18:48

the algorithm is also able to do the generation ...

18:51

Well, it's a question about how much intelligence is behind

18:53

it, right? The content

18:55

is being generated by, let's say, one humanity

18:57

worth of intelligence, and you can quantify a humanity.

19:01

It's exa-flops,

19:04

yada-flops, but you can quantify

19:07

it.

19:07

Once that generation is being done by 100 humanities,

19:10

you're done.

19:11

It's actually

19:14

scale, that's the problem, but

19:16

also speed.

19:19

Yeah. What if

19:22

it's manipulating the

19:24

very limited human dopamine engine

19:27

for porn? Imagine it's just TikTok,

19:30

but for porn.

19:32

That's like a brave new world. I

19:34

don't even know what it'll look like, right? Again,

19:37

you can't imagine the behaviors of something

19:39

smarter than you, but a

19:41

super intelligent ... An agent

19:44

that just dominates your intelligence so much

19:46

will be able to completely manipulate

19:49

you. So that it

19:51

won't really manipulate, it'll just move past

19:53

us? It'll just kind of exist

19:56

the way water exists or the air exists?

19:59

You see? And that's the whole AI

20:01

safety thing. It's

20:03

not the machine that's going to do that.

20:05

It's other humans using the machine that are going to do that

20:08

to you. Yeah. Because

20:10

the machine is not interested in hurting humans. The

20:13

machine is a machine. But the human

20:15

gets the machine, and there's a lot of humans out

20:17

there very interested in manipulating you.

20:20

Well, let me bring up Eliezer

20:23

Yatkowsky, who

20:25

recently sat where you're sitting. He

20:28

thinks that AI will almost surely

20:30

kill everyone.

20:32

Do you agree with him or not?

20:35

Yes, but maybe for a different reason. Okay.

20:40

And then I'll try to

20:42

get you to find hope, or

20:45

we could find a note to that answer. But

20:47

why yes? Okay. Why

20:50

didn't nuclear weapons kill everyone? That's

20:51

a good question. I think there's an answer. I

20:54

think it's actually very hard to deploy nuclear weapons tactically.

20:58

It's very hard to accomplish tactical objectives.

21:00

Great. I can nuke their country. I have

21:02

an irradiated pile of rubble. I don't want

21:05

that. Why not?

21:06

Why don't I want an irradiated pile of rubble? For

21:09

all the reasons no one wants an irradiated pile of rubble.

21:12

Because you can't use that land for

21:14

resources. You

21:16

can't populate the land. Yeah. What you

21:18

want a total victory in a

21:20

war is not usually the irradiation

21:24

and eradication of the people there. It's the

21:26

subjugation and domination of the people.

21:29

Okay. So you can't

21:31

use this strategically tactically in a war

21:34

to help gain a military advantage.

21:39

It's all complete destruction. But

21:42

there's egos involved. It's still surprising. It's

21:44

still surprising that nobody pressed the big red button.

21:47

It's somewhat surprising. But you

21:50

see, it's the little red button that's going to be pressed

21:52

with AI. That's going

21:55

to... And that's why we die. It's

21:57

not because the AI... If

22:00

there's anything in the nature of AI, it's just the

22:02

nature of humanity. What's the algorithm behind

22:04

the little red button? What

22:07

possible ideas do you have for how a

22:09

human species ends? Sure. So I

22:11

think the most

22:14

obvious way to me is wireheading.

22:16

We end up amusing ourselves to death.

22:20

We end up all

22:21

staring at that infinite TikTok and

22:23

forgetting to eat.

22:26

Maybe it's even more benign than this. Maybe

22:28

we all just stop reproducing.

22:32

To be fair, it's probably

22:34

hard to get all of humanity. Yeah. The

22:40

interesting thing about humanity is the diversity

22:43

in it. Oh, yeah. Organisms in general. There's

22:45

a lot of weirdos out there.

22:47

Two of them are sitting here. I mean, diversity

22:50

in humanity is- We do respect. I

22:53

wish I was more weird. No, like, I'm

22:56

kind of, look, I'm drinking smart water, man. It's like a Coca-Cola

22:58

product, right? You want corporate, George

23:00

Haas. I want corporate. No,

23:02

the amount of diversity in humanity, I think, is decreasing.

23:05

Just like all the other biodiversity on the planet.

23:08

Oh, boy. Yeah. Social media

23:10

is not helping, huh? Go eat McDonald's in China.

23:12

Yeah. No,

23:15

it's the interconnectedness that's

23:17

doing it. Oh,

23:20

that's interesting. So everybody starts

23:22

relying on the connectivity of

23:24

the internet. And over time,

23:26

that reduces the intellectual diversity. And

23:29

then that gets everybody into a

23:31

funnel. There's still going to be a guy in Texas.

23:34

There is. And yeah. A bunker. To

23:36

be fair, do I think AI kills us all?

23:39

I think AI kills everything we call society

23:42

today. I do not think it actually kills

23:44

the human species. I think that's actually incredibly

23:46

hard to do.

23:48

Yeah, but society, if we start

23:50

over, that's tricky. Most of us don't know how

23:53

to do most things. Yeah, but some

23:55

of us do. And they'll be

23:57

OK, and they'll rebuild after the.

24:00

Great AI What's

24:02

rebuilding look like how far like how

24:05

much do we lose? What

24:07

is human civilization done? That's

24:09

interesting the combustion engine electricity

24:13

so Power

24:15

and energy that's interesting Like

24:18

how to harness energy? Well,

24:20

they're gonna be religiously against that Are

24:24

they going to get back to like fire?

24:27

Sure. I mean, they'll be a they'll be a little

24:30

bit like, you know some kind of Amish looking

24:32

kind of thing I think I think they're going to have very strong

24:34

taboos against technology

24:37

hmm like technology

24:39

is almost like a new religion technology is the devil

24:41

and Nature

24:44

is God

24:46

Sure, so closer to nature, but can

24:48

you really get away from AI if it destroyed 99%

24:50

of the human species, isn't it? Somehow

24:53

have a hold like a stronghold.

24:56

What's interesting about

24:58

Everything we build I think we are going to build super

25:00

intelligence before we build any sort of

25:03

robustness in the AI We

25:05

cannot build an AI that is capable

25:07

of going out into nature and surviving like

25:10

a Like a bird, right?

25:13

A bird is an incredibly robust

25:15

Organism we've built nothing like this. We haven't built

25:18

a machine that's capable of reproducing

25:20

Yes, but There's

25:23

you know, I work with leg robots a lot now.

25:25

I have a bunch of them

25:28

They're mobile That

25:31

can't reproduce but all they

25:33

need is I guess you're saying they can't repair

25:35

themselves But if you have a large number if you have

25:37

like a hundred million of them, let's just focus

25:39

on them reproducing, right? Do they have microchips

25:41

in them? Okay, then

25:43

do they include a fab?

25:46

No, then how are they gonna reproduce the

25:48

other day? It doesn't have to

25:50

be all on board, right? They

25:52

can go to a factory to a repair

25:54

shop. Yeah, but then you're really moving

25:57

away from robustness. Yes all

25:59

of life

25:59

is capable of reproducing without needing

26:02

to go to a repair shop. Life

26:04

will continue to reproduce in the complete absence

26:06

of civilization.

26:08

Robots will not. So when

26:11

the, if the AI apocalypse

26:13

happens, I

26:14

mean the AI's are going to probably die out because

26:17

I think we're going to get, again, super intelligence long before

26:19

we get robustness.

26:20

What about if you just improve the

26:23

fab to where you just

26:26

have a 3D printer that can always help you? Well,

26:29

that'd be very interesting. I'm interested in building that. Of

26:32

course you are. You think, how difficult is that

26:34

problem to have a robot that

26:38

basically can build itself?

26:40

Very, very hard. I think you've mentioned

26:43

this, like, to me

26:45

or somewhere where people

26:47

think it's easy conceptually. And

26:50

then they remember that you're going to have to have a fab.

26:53

Yeah, on board. Of course. So

26:56

3D printer that prints a 3D printer.

26:59

Yeah.

27:00

Yeah, on legs. Why

27:02

is that hard? Well, because it's not, I

27:04

mean a 3D printer is a very simple

27:07

machine, right? Okay, you're going to print chips,

27:09

you're going to have an atomic printer, how are you going to dope

27:12

the silicon? Yeah.

27:14

Right? How are you going to etch the silicon? You're

27:17

going to have to have a

27:19

very interesting kind of fab if you want to have

27:22

a lot of computation on board. But

27:24

you can do, like, structural

27:28

type of robots that are dumb.

27:30

Yeah, but structural type of robots

27:33

aren't going to have the intelligence required to survive

27:35

in any complex environment.

27:36

What about, like, ants type of systems? We

27:39

have, like, trillions of them. I don't

27:41

think this works. I mean, again, like, ants

27:43

at their very core are made up of cells

27:45

that are capable of individually reproducing.

27:48

They're doing quite a lot of computation

27:50

that we're taking for granted. It's not even just

27:52

the computation. It's that reproduction is so

27:55

inherent. Okay, so, like, there's two stacks of life in

27:57

the world. There's the biological

27:59

stack and the silicon stack.

27:59

The biological stack

28:02

starts with reproduction. Reproduction

28:05

is at the absolute core. The first proto

28:08

RNA organisms were capable of reproducing. The

28:11

silicon stack, despite as far

28:13

as it's come, is nowhere near

28:15

being able to reproduce. Yeah.

28:18

So the fab movement, digital

28:23

fabrication,

28:25

fabrication in the full range of what that means is

28:28

still in the early stages. Yeah.

28:30

You're interested in this world. Even

28:32

if you did put a fab on the machine, let's say, okay, we can build

28:35

fabs. We know how to do that as humanity. We

28:37

can probably put all the precursors that build all the machines

28:40

and the fabs also in the machine. So first off, this machine is

28:42

going to be absolutely massive.

28:44

I mean, we almost have a... Think

28:47

of the size of the thing required to reproduce

28:49

a machine today.

28:52

Is our civilization capable of reproduction?

28:55

Can

28:56

we reproduce our civilization on Mars?

29:00

If we were to construct a machine that is made up of humans, like

29:02

a company, it can reproduce

29:04

itself. Yeah. I don't know. It

29:08

feels like 115 people.

29:12

I think it's so much harder than that. Let's

29:16

see. I

29:18

believe that Twitter can be run by 50 people.

29:22

I think that this is going to take most

29:24

of... It's just most

29:27

of society. We live in one globalized

29:29

world. No, but you're not interested in running Twitter.

29:32

You're interested in seeding.

29:33

You want to seed

29:36

a civilization then because humans

29:38

can have sex. Yeah,

29:40

okay. So you're talking about the humans reproducing

29:42

and basically, what's the smallest self-sustaining colony

29:44

of humans? Yeah. Yeah, okay, fine. Over

29:48

time, they will. I think you're being...

29:51

We have to expand our conception of time

29:53

here. Come back to the original. Time

29:56

scale. I mean, over across...

29:59

maybe a hundred generations, we're back to making

30:02

chips. No? If you seed

30:04

the colony correctly.

30:06

Maybe. Or maybe they'll watch our

30:08

colony die out over here and be like, we're

30:10

not making chips, don't make chips. No, but you

30:12

have to seed that colony correctly. Whatever you

30:14

do, don't make chips. Chips are what

30:17

led to their downfall.

30:20

Well, that is the thing that humans do. They

30:22

come up, they construct a devil, a

30:24

good thing and a bad thing, and they really stick by that,

30:27

and then they murder each other over that. There's always

30:29

one asshole in the room who murders everybody. And

30:33

he usually makes tattoos and nice branding. Do

30:35

you need that asshole? That's the question, right?

30:38

Humanity works really hard today to get rid of that asshole,

30:40

but I think they might be important. Yeah,

30:43

this whole freedom of speech thing, it's

30:45

the freedom of being an asshole seems kind of important.

30:47

That's right.

30:49

Man, this thing, this fab, this human

30:51

fab that we've constructed, this human

30:53

civilization, is pretty interesting. And now

30:56

it's

30:56

building artificial copies

30:59

of itself, or artificial copies of

31:01

various aspects of itself

31:04

that seem interesting, like intelligence. And

31:07

I wonder where that goes. I

31:10

like to think it's just like another stack for life. Like

31:12

we have the biostack life, like we're a biostack life,

31:14

and then the silicon stack life. But it seems

31:16

like the ceiling, or there might

31:18

not be a ceiling, or at least the ceiling is much higher

31:21

for the silicon stack. Oh,

31:23

no, we don't know what the ceiling is for the biostack

31:26

either. The biostack just

31:28

seemed to move slower. You have

31:31

Moore's law, which is not dead

31:33

despite many proclamations. In

31:35

the biostack or the silicon stack? In the silicon stack. And

31:38

you don't have anything like this in the biostack. So I have

31:40

a meme that I posted, I tried to make a

31:42

meme, it didn't work too well. But I posted

31:44

a picture of Ronald Reagan and Joe

31:46

Biden, and you look, this is 1980 and this is 2020. And

31:50

these two humans are basically like the same, right?

31:52

There's been no change in humans in the

31:55

last 40 years.

32:00

And then I posted a computer from 1980 and a computer

32:02

from 2020. Wow. Yeah,

32:07

with the early stages, right? Which

32:09

is why you said when you said the fab,

32:11

the size of the fab required to make another

32:13

fab is like

32:16

very large right now. Oh,

32:18

yeah. But computers were very large 80 years

32:23

ago and they got pretty

32:26

tiny. People

32:29

are starting to want to wear them on their face in

32:33

order to escape reality. That's

32:36

the thing in order to be live inside

32:38

the computer. Yeah. Put a

32:40

screen right here. I don't have to see the rest

32:42

of you assholes. I've been ready for a long

32:44

time. You like virtual reality? I love

32:46

it. Do you want to live

32:49

there? Yeah. Yeah.

32:52

Part of me does too. How far away are

32:54

we, do you think? Anything

32:57

from what you can buy today far? Very

33:00

far. I got to tell you that

33:02

I had the experience of

33:06

Meta's Kodak avatar where

33:09

it's an ultra high resolution

33:12

scan. It looked

33:15

real. I mean, the

33:17

headsets just are not quite at like eye resolution

33:19

yet. I haven't put on any

33:22

headset where I'm like, oh, this could be

33:24

the real world. Whereas when

33:26

I put good headphones on, audio

33:27

is there. We can

33:30

reproduce audio that I'm like, I'm actually in a jungle right

33:32

now. If I close my eyes, I can't tell I'm not. Yeah.

33:36

But then there's also smell and all that kind of stuff. Sure.

33:39

I don't know. The power

33:41

of imagination or the power of the

33:43

mechanism in the human mind that fills the gaps

33:46

that kind of reaches and wants to make

33:48

the thing you see in the virtual world real

33:51

to you, I

33:53

believe in that power. Or humans

33:55

want to believe. Yeah.

33:58

What if you're lonely? What if you're sad? What

34:00

if you're really struggling in life and

34:02

here's a world where you don't have to struggle

34:05

anymore Humans want to believe so much

34:07

that people think the large language models are conscious.

34:10

That's how much humans want to believe

34:12

Strong words. He's throwing

34:14

left and right hooks. Why do you

34:17

think large language models are not conscious? I

34:19

don't think I'm conscious. Oh,

34:21

so what is consciousness then George Hans?

34:24

It's like

34:25

what it seems to mean to people. It's just like

34:27

a word that atheists use for souls Sure,

34:31

but that doesn't mean soul is not an interesting word

34:34

If consciousness is a spectrum. I'm definitely way

34:37

more conscious than the large language models are

34:39

I

34:41

Think the large language models are less conscious than

34:43

a chicken When is

34:45

the last time you see a chicken? In

34:48

Miami like a couple months ago. How

34:52

no like a living chicken living chickens walking

34:54

around Miami It's crazy like on the street.

34:56

Yeah, like a chicken chicken All

35:02

right, I was trying to call you all like

35:04

a good journalist and I I got shut

35:06

down Okay,

35:09

but

35:10

you don't think

35:12

much about this kind

35:15

of Subjective

35:18

feeling that it feels like something

35:21

to exist and then as an observer

35:25

you can Have a sense

35:27

that an entity is not only

35:29

intelligent but has a kind of

35:33

Subjective experience of its reality

35:35

like a self-awareness That is capable

35:38

of like suffering of hurting of being excited

35:40

by the environment in a way. That's not

35:42

merely

35:44

Kind of an artificial response but a deeply

35:47

felt one Humans want to believe so

35:50

much that if I took a rock and a sharpie

35:52

and drew a sad face on the rock They'd think the

35:54

rock is sad

35:55

Yeah,

35:58

and you're saying when we look in the mirror we we

36:00

apply the same smiley face with rock.

36:02

Pretty much, yeah. Isn't that weird though?

36:05

You're not conscious?

36:09

But you do believe in consciousness. Really?

36:12

It's just, it's unclear. Okay, so to you it's like

36:14

a little, like a symptom of the bigger

36:16

thing that's not that important. Yeah, I mean

36:18

it's interesting that like human systems

36:21

seem to claim that they're conscious. And I guess

36:23

it kind of like says something and they straight up like,

36:25

okay, what do people mean? Even if you don't believe

36:27

in consciousness, what do people mean when they say consciousness?

36:30

And there's definitely like

36:31

meanings to it. What's your favorite

36:33

thing to eat? Pizza.

36:37

Cheese pizza, what are the toppings? I like cheese pizza.

36:40

Don't say pineapple. No, I don't like pineapple. Okay,

36:42

pepperoni pizza. Has they put any ham on it? Oh, that's real

36:44

bad. What's the best pizza? What

36:47

are we talking about here? Like, do you like cheap crappy pizza?

36:50

Chicago deep dish cheese pizza. Oh,

36:52

that's my favorite. There you go, you bite into a deep

36:54

dish, a cargo deep dish pizza,

36:56

and it feels like you were starving. You haven't

36:59

eaten for 24 hours. You just bite

37:01

in and you're hanging out with somebody that

37:03

matters a lot to you and you're there with the pizza. Sounds

37:05

real nice, huh? Yeah, all right. It

37:08

feels like something. I'm George

37:10

motherfucking hot eating a fucking

37:13

Chicago deep dish pizza. There's

37:15

just a full peak

37:17

living experience of

37:20

being human, the top of the human condition.

37:23

Sure. It feels like something

37:25

to experience that. Why

37:28

does it feel like something? That's consciousness,

37:31

isn't it? If that's the word

37:33

you want to use to describe it, sure. I'm not going to deny

37:35

that that feeling exists. I'm not going to deny that I experienced

37:38

that feeling. When, I

37:40

guess what I kind of take issue to is that

37:42

there's some like, like how

37:44

does it feel to be a web server? Do 404s hurt?

37:49

Not yet. How would you know what suffering

37:51

looked like? Sure, you can recognize a suffering

37:53

dog because we're the same stack as the dog. All

37:56

the bio stack stuff, especially mammals,

37:59

it's really easy. You can... Game

38:02

recognizes game. Yeah. Versus

38:04

the silicon stack stuff. It's like, you

38:06

have no idea.

38:08

You have... Wow, the little thing

38:10

has learned to mimic.

38:15

But then I realized that that's all we are too.

38:18

Oh look, the little thing has learned to mimic. Yeah.

38:21

I guess, yeah, 404 could be suffering, but it's so far from our kind

38:23

of living.

38:27

Living

38:30

organism, our kind of stack. But

38:32

it feels like AI can start

38:35

maybe mimicking the biological

38:37

stack better, better, better. Because it's trained. Retrained

38:39

it, yeah. And so, in

38:41

that, maybe that's the definition of consciousness. Is

38:44

the bio stack consciousness? The definition

38:46

of consciousness is how close something looks to human.

38:48

Sure, I'll give you that one.

38:50

No, how close something

38:52

is to the human experience. Sure.

38:55

It's a very anthropocentric

38:58

definition, but... That's all we got. Sure.

39:01

No, and I don't mean to like... I think there's

39:03

a lot of value in it. Look, I just started my second company.

39:05

My third company will be AI Girlfriends.

39:08

No, like I mean it. I want to find out what your fourth company

39:11

is after that. Oh wow. Because I think once

39:13

you have AI Girlfriends, it's... Oh

39:17

boy. Does it get interesting?

39:20

Well, maybe let's go there. I mean, the relationships

39:22

with AI, that's creating human-like

39:24

organisms, right?

39:27

And part of being human is being conscious, is

39:30

having the capacity to suffer, having the capacity

39:32

to experience this life richly in

39:34

such a way that you can empathize...

39:36

The AI system can empathize with you

39:38

and you can empathize with it. Or you can

39:40

project your anthropomorphic

39:44

sense of what the other entity is experiencing.

39:48

And an AI model would need

39:50

to create that experience inside your mind. And

39:53

it doesn't seem that difficult. Yeah, but okay.

39:55

So here's where it actually gets totally different, right?

39:59

When you interact... with another human, you can

40:01

make some assumptions.

40:04

When you interact with these models, you can't. You

40:06

can make some assumptions that that other human experiences

40:09

suffering and pleasure in a pretty

40:11

similar way to you do. The golden rule applies.

40:15

With an AI model, this isn't really true.

40:18

These large language models are good at fooling people

40:20

because they were trained on a whole bunch

40:23

of human data and told to mimic it.

40:25

But if the AI system says,

40:28

hi, my name is Samantha,

40:31

it has a backstory. I went to college

40:33

here and there. Maybe you'll integrate

40:36

this in the AI system. I made some chatbots.

40:38

I gave them backstories. It was lots of fun. I

40:40

was so happy when Llama came out. Yeah. We'll

40:43

talk about Llama. We'll talk about all that. But like,

40:45

you know, the rock with the smiley face. It

40:49

seems pretty natural for you to anthropomorphize

40:51

that thing and then start dating it. Before

40:55

you know it, you're married and

40:58

have kids. With a rock? With

41:00

a rock. And there's pictures on Instagram

41:02

with you and a rock and a smiley face. To

41:04

be fair, like, you know, something that people generally look

41:06

for in the look of someone to date is intelligence

41:08

in some

41:10

form. And the rock doesn't really have

41:12

intelligence. Only a pretty desperate person would date a rock.

41:16

I think we're all desperate deep down. Oh,

41:18

not rock level desperate. All

41:20

right. Not

41:23

rock level desperate,

41:26

but AI level

41:28

desperate. I don't know. I think all

41:30

of us have a deep loneliness. It just feels

41:32

like the language models are there. Oh,

41:35

I agree. And you know what? I won't even say this so cynically.

41:37

I will actually say this in a way that like, I want AI

41:39

friends. I do. Yeah. Like,

41:42

I would love to. You know, again, the

41:44

language models now are still a little like,

41:48

people are impressed with these GPT things.

41:50

And I look at like, or like, or

41:53

the copilot, the coding one. And

41:55

I'm like, okay, this is like junior engineer

41:57

level. And these people are like fiverr level.

41:59

artists and copywriters.

42:02

Like, okay, great, we got like

42:04

Fiverr and like junior engineers, okay,

42:06

cool. Like, and this is just the start

42:09

and it will get better, right? Like

42:11

I can't wait to have AI friends who

42:13

are more intelligent than I am.

42:15

So Fiverr is just a temporary, it's not the

42:17

ceiling. No, definitely not. Is

42:21

it countless cheating

42:23

when you're talking to an AI model, emotional

42:26

cheating? That's

42:30

up to you and your human partner to

42:32

define. Oh, you have to, all right. Yeah,

42:35

you have to have that conversation, I guess.

42:37

All right, I mean, integrate that

42:40

with porn and all this stuff. No,

42:42

I mean, it's similar kind of to porn. Yeah. Yeah. Right,

42:45

I think people in relationships have different views on that.

42:47

Yeah, but most people don't

42:50

have like serious

42:54

open conversations about all the different

42:57

aspects of what's cool and what's not. And

43:00

it feels like AI is a really weird conversation

43:02

to have. The

43:04

porn one is a good branching off point. Like

43:06

these things, you know, one of my scenarios that I put in my chatbot

43:09

is I go, you know, a

43:12

nice girl named Lexi, she's 20, she just moved

43:14

out to LA, she wanted to be an actress, but she

43:16

started doing OnlyFans instead and you're on a date with her,

43:18

enjoy. Oh,

43:22

man. Yeah,

43:24

and so is that if you're actually dating somebody

43:26

in real life, is that cheating? I

43:29

feel like it gets a little weird. Sure. It gets

43:31

real weird. It's like, what are you allowed to

43:33

say to an AI bot? Imagine having

43:35

that conversation with a significant other. I mean,

43:37

these are all things for people to define in their relationships.

43:40

What it means to be human is just gonna start to

43:42

get weird. Especially online.

43:44

Like, how do you know? Like, there'll

43:46

be moments when you'll have what you

43:48

think is a real human you interacted

43:50

with on Twitter for years and you realize it's not.

43:53

I spread, I love this meme,

43:56

heaven banning.

43:57

Tell you about shadow banning. Yeah. Shadow

44:00

banning, okay, you post, no one can see it. Heaven

44:02

banning, you post, no one can see

44:04

it, but a whole lot of AIs are spot

44:06

up to interact with you. Well,

44:10

maybe that's what the way human civilization ends

44:12

is all of us are heaven banned. There's

44:15

a great, it's called My Little Pony

44:17

Friendship is Optimal.

44:18

It's a sci-fi story that explores

44:21

this idea. Friendship is optimal. Friendship

44:24

is optimal. Yeah, I'd like to have some, at least

44:26

on the intellectual realm, from AI friends

44:29

that argue with me.

44:30

But the romantic realm

44:33

is weird, definitely weird.

44:38

But not out of the realm of the

44:41

kind of weirdness that human civilization is capable

44:44

of, I think. I want

44:46

it. Look, I want it. If no one else wants

44:48

it, I want it. Yeah, I think a lot of people probably

44:50

want it. There's a deep loneliness.

44:53

And I'll feel their loneliness

44:56

and, you know, it's just, we'll only advertise to you

44:58

some of the time. Yeah, maybe the conceptions

45:00

of monogamy change too. Like, I grew up

45:02

in a time, like, I value monogamy, but maybe

45:04

that's a silly notion when you have

45:07

arbitrary number of AI systems. Mm,

45:10

this interesting path from rationality

45:13

to polyamory, that doesn't make sense

45:15

for me. For you. But you're just

45:17

a biological organism who was born before,

45:20

like, the internet really took off.

45:23

The crazy thing is, like,

45:25

culture is whatever we define it as.

45:28

These things are not, like,

45:31

is a lot problem in moral philosophy, right? There's

45:33

no, like, okay, what is might be that, like,

45:36

computers are capable of mimicking, you

45:39

know, girlfriends perfectly. They pass the girlfriend Turing

45:41

test, right? But that doesn't say anything about

45:43

a lot. That doesn't say anything about how we ought to

45:45

respond to them as a civilization. That doesn't say we ought

45:47

to get rid of monogamy, right? That's a

45:50

completely separate question, really a religious

45:52

one.

45:52

And Turing test, I wonder

45:54

what that looks like. Girlfriend Turing test. Are you

45:57

writing that? Will

45:59

you be the... the Alan Turing of the

46:01

21st century that writes the girlfriend Turing

46:03

test paper? No, I mean, of course, my, hey, girlfriends,

46:06

their goal is to pass the girlfriend Turing test.

46:09

No, but there should be like a paper that kind

46:11

of defines the test. Or,

46:14

I mean, the question is if it's deeply personalized

46:16

or there's a common thing that really gets everybody.

46:21

Yeah, I mean, you know, look, we're a company,

46:23

we don't have to get everybody, we just have to get a large enough clientele

46:26

to stay. I like how you're already thinking

46:28

company. All right, let's, before

46:30

we go to company number three and company number

46:32

four, let's go to company number two. All right.

46:35

TinyCorp,

46:37

possibly one of the greatest names of all

46:39

time for a company. You've

46:41

launched a new company called

46:43

TinyCorp that leads the development of TinyGrad.

46:46

What's the origin story

46:48

of TinyCorp and TinyGrad?

46:50

I started TinyGrad as

46:53

like a toy project, just to teach myself,

46:55

okay, like, what is a convolution? What

46:58

are all these options you can pass to them? What is

47:00

the derivative of a convolution, right? Very similar

47:03

to Carpathi wrote MicroGrad. Very

47:06

similar. And

47:08

then

47:09

I started realizing, I started thinking

47:11

about like

47:12

AI chips. I started thinking about chips that

47:14

run AI and I

47:17

was like, well, okay, this is going to be

47:19

a really big problem. If Nvidia

47:22

becomes a monopoly here, how

47:25

long before Nvidia is nationalized?

47:28

So you, one

47:30

of the reasons that start

47:33

TinyCorp is to challenge Nvidia.

47:36

It's not so much

47:37

to challenge Nvidia. I actually, I

47:39

like Nvidia and it's to make

47:42

sure power

47:45

stays decentralized. Yeah.

47:48

And here's computational

47:50

power. I see you

47:52

Nvidia is kind of locking down the

47:54

computational power of the world.

47:56

If Nvidia becomes

47:58

just like 10X better than everything.

47:59

else, you're giving a big advantage

48:02

to somebody who can secure NVIDIA

48:05

as a resource. Yeah.

48:07

In fact,

48:08

if Jensen watches this podcast, he may want to consider

48:11

this.

48:12

He may want to consider making sure his company is not

48:14

nationalized. Do

48:16

you think that's an actual threat? Oh, yes. No,

48:20

but there's so much, you know, there's

48:23

AMD. So we have NVIDIA and AMD,

48:25

great. All right.

48:27

You don't think there's like a push

48:31

towards like selling, like Google

48:33

selling TPUs or something like this? You

48:35

don't think there's a push for that? Have you seen it? Google

48:37

loves to rent you TPUs. It

48:39

doesn't. You can't buy it at Best Buy?

48:42

No.

48:43

So I started work on a chip. I

48:46

was

48:48

like, okay, what's it going to take to make a chip? And

48:51

my first notions were all completely wrong about why,

48:53

about like how you could improve on GPUs. And

48:56

I will take this. This is from Jim

48:58

Keller on your podcast. And

49:00

this is one of my absolute favorite

49:03

descriptions of computation.

49:05

So there's three kinds of computation paradigms

49:08

that are common in the world today.

49:10

They're CPUs. And

49:12

CPUs can do everything. CPUs can do

49:14

add and multiply.

49:15

They can do load and store, and they can do

49:17

compare and branch. And when I say they can

49:19

do these things, they can do them all fast, right? So

49:22

compare and branch are unique to CPUs. And

49:24

what I mean by they can do them fast is they can do things like

49:27

branch prediction and speculative execution. And they

49:29

spend tons of transistors and they use like super deep

49:31

reorder buffers in order to make these things

49:33

fast. Then you have a simpler computation

49:36

model GPUs. GPUs can't really do

49:38

compare and branch. I mean, they can, but it's horrendously

49:40

slow. But GPUs can do arbitrary

49:42

load and store. GPUs can do things

49:45

like X, D reference Y.

49:47

So they can fetch from arbitrary pieces of memory. They

49:49

can fetch from memory that is defined by the contents of the data.

49:52

The third model of computation

49:54

DSPs and DSPs are just add and

49:56

multiply.

49:57

Like they can do load and store, but only static load.

49:59

stores, only loads and stores that are known before

50:02

the program runs. And you look at neural

50:04

networks today and 95% of neural networks

50:06

are all the DSP paradigm,

50:09

they are just statically scheduled

50:12

ads and multiplies. So

50:14

TinyGuard really took this idea and,

50:17

and I'm still working on it to extend this as

50:19

far as possible. Um, every

50:21

stage of the stack has Turing completeness, right? Python

50:24

has Turing completeness. And then we take Python and

50:26

we go into C plus plus, which is Turing complete. And

50:28

maybe C plus plus calls into some CUDA kernels,

50:30

which are turn complete, the CUDA kernels go through LLVM,

50:33

which is turn complete into PTX, which is turn complete

50:35

to SAS, which is turn complete on a current turn complete processor.

50:37

I want to get Turing completeness out of the stack entirely.

50:40

Because once you get rid of Turing completeness, you can reason about

50:42

things. Rice's theorem and the halting problem

50:45

do not apply to admiral machines.

50:47

Okay. What's

50:50

the power and the value of getting Turing completeness out

50:52

of, out of, are we talking about

50:54

the hardware or the software? Every

50:57

layer of the stack, every layer

50:59

of the stack, removing turn completeness allows

51:01

you to reason about things, right? So

51:03

the reason you need to do branch prediction in a CPU

51:06

and the reason it's prediction and the branch predictors are,

51:08

I think they're like 99% on CPUs. Why

51:10

did they get 1% of them wrong? Well, they

51:12

get 1% wrong because you can't

51:15

know.

51:15

Right. That's the halting problem. It's equivalent

51:18

to the halting problem to say whether a branch is going

51:20

to be taken or not. Um, I can

51:22

show that, but

51:24

the. Admiral

51:26

machine, the neural network runs

51:28

the identical compute every time. The

51:31

only thing that changes is the data. So

51:34

when you realize this, you think about, okay,

51:37

how can we build a computer? How can we build

51:39

a stack that takes maximal advantage of

51:41

this idea? Uh,

51:43

so

51:44

what makes tiny grad different from other

51:46

neural network libraries is it does not have

51:49

a primitive operator even for matrix multiplication.

51:52

And this is every single one. They

51:54

even have primitive operators for things like convolutions.

51:56

So no mat mall. No, Matt. Well,

51:59

here's what a.

51:59

I'm at my list, so I'll use my hands to talk here.

52:02

So if you think about a cube, and I put my two

52:04

matrices that I'm multiplying on two faces of the

52:07

cube, you can

52:09

think about the matrix multiply as, okay,

52:12

the n cubed, I'm going to multiply for

52:14

each one in the cubed, and then I'm going to do a sum, which

52:16

is a reduce up to here to the third

52:18

face of the cube, and that's your multiplied matrix. So

52:21

what a matrix multiply is, is a bunch of shape

52:24

operations, right? A bunch of permutes,

52:26

reshapes, and expands on the two matrices.

52:29

A multiply, n cubed, a

52:31

reduce, n cubed, which gives

52:33

you an n squared matrix. Okay, so

52:36

what is the minimum number of operations that can accomplish

52:38

that if you don't have MatMall

52:40

as a primitive? So TinyGrad has

52:43

about 20, and you can compare TinyGrad's

52:46

op-set or IR to things like XLA

52:49

or PrimTorch. So XLA and

52:51

PrimTorch are ideas where like, okay, Torch

52:54

has like 2000 different kernels,

52:58

PyTorch 2.0 introduced PrimTorch,

53:00

which has only 250.

53:01

TinyGrad has order of magnitude 25.

53:05

It's 10X less than

53:07

XLA or PrimTorch. And

53:09

you can think about it as kind of like RISC versus CISC,

53:12

right?

53:13

These other things are CISC-like

53:15

systems. TinyGrad is RISC.

53:18

And RISC-1. RISC architecture

53:21

is going to change everything. 1995 hackers. Wait,

53:25

really? That's an actual thing? Angelina

53:27

Jolie delivers the line, RISC architecture

53:29

is going to change everything in 1995. Wow. And

53:32

here we are with ARM in the phones and

53:34

ARM everywhere. Wow, I

53:36

love it when movies actually have real things in

53:38

them. Right. Okay, interesting.

53:41

So you're thinking of this as the RISC

53:44

architecture of ML

53:45

Stack. 25, huh?

53:49

What can you go through the four

53:51

op types?

53:56

Okay, so you have UnaryOps, which

53:59

take in... a Tensor

54:02

and return a tensor of the same size and do some

54:04

unary opt to it X log Reciprocal

54:08

sign right they take in one and their point

54:10

wise Really you

54:13

yeah really you almost all activation

54:15

functions are unary ops Some

54:17

combinations of unary ops together

54:19

is still a unary op

54:22

Then you have binary ops binary ops are like

54:24

point wise addition multiplication division compare

54:28

It takes in two tensors of equal size and

54:31

outputs one tensor Then

54:33

you have reduce ops Reduce

54:36

ops will like take a three-dimensional tensor and

54:38

turn it into a two-dimensional tensor Or

54:40

three-dimensional tensor turn it to zero dimensional tensor

54:42

think like a sum or max

54:44

are really the common ones there And

54:47

then the fourth type is movement ops and

54:50

movement ops are different from the other types because they don't actually

54:52

require computation They require different ways to look

54:54

at memory So that includes reshapes

54:57

permutes Expans flips

55:00

does the main ones probably so with that you have enough

55:02

to make a map mall and convolutions

55:05

and every convolution you can Imagine dilated

55:07

convolutions strided convolutions transposed

55:10

convolutions

55:12

You're right on github about laziness

55:16

showing a map mall Matrix

55:18

multiplication see how despite the style

55:20

is fused into one kernel with

55:23

the power of laziness Can you elaborate

55:25

on this power of laziness sure so if

55:27

you type in Pytorch a times

55:30

B plus C?

55:32

What this is going to do is

55:34

it's going to first multiply add and

55:36

be a and B and store that result

55:38

into memory

55:39

and then it is going to add C by reading

55:42

that result from memory reading C for memory

55:44

and writing that out to memory There

55:47

is way more loads and stores to memory than

55:49

you need there If you don't actually do

55:52

a times B as soon as you see it

55:54

if you wait

55:55

Until the user actually realizes that

55:58

tensor until the laziness actually resolved You

56:00

can fuse that plus C. This is like,

56:03

it's the same way Haskell works. So

56:05

what's the process of porting a model

56:08

into TinyGrad? So TinyGrad's

56:10

front end looks very similar to PyTorch. I

56:13

probably could make a perfect,

56:15

or pretty close to perfect interop layer if I

56:17

really wanted to. I think that there's some things that

56:19

are nicer about TinyGrad syntax than PyTorch, but

56:22

the front end looks very Torch-like. You can also

56:24

load in Onyx models. We have

56:27

more Onyx tests passing than Core ML. Core

56:30

ML. Okay, so- We'll pass Onyx

56:32

runtime soon. What about the

56:34

developer experience with TinyGrad? What

56:38

it feels like versus PyTorch?

56:40

By the way, I

56:43

really like PyTorch. I think that it's actually a very

56:45

good piece of software. I think that they've

56:47

made a few different trade-offs, and

56:49

these different trade-offs are

56:52

where, TinyGrad takes

56:54

a different path. One of the biggest differences

56:56

is it's really easy to see the kernels

56:59

that are actually being sent to the GPU.

57:00

If you run PyTorch on

57:02

the GPU,

57:04

you do some operation, and you don't know what

57:06

kernels ran. You don't know how many kernels ran. You

57:08

don't know how many flops were used. You don't know how much

57:10

memory accesses were used. TinyGrad

57:12

type debug equals two, and it

57:14

will show you in this beautiful style every

57:17

kernel that's run.

57:19

How many flops

57:21

and how many bytes. So

57:24

can you just linger

57:26

on what problem

57:28

TinyGrad solves? TinyGrad

57:30

solves the problem of porting new ML

57:32

accelerators quickly. One

57:34

of the reasons, tons

57:37

of these companies now, I think Sequoia

57:40

marked Graphcore to zero.

57:42

Seribis, TenzTorrent,

57:44

Grok, all of

57:46

these ML accelerator companies,

57:49

they built chips. The chips were good. The

57:51

software was terrible. And

57:54

part of the reason is because I think the same problem is happening

57:56

with Dojo. It's really, really

57:58

hard to write a PyTorch port.

57:59

because you have to write 250 kernels

58:03

and you have to tune them all for performance. What

58:06

does Jim Keller think about Tanya Grad?

58:08

You guys hung on

58:10

quite a bit, so he's, you know, he

58:13

was involved with Tencentorrent. What's

58:15

his praise and what's his criticism

58:18

of what you're doing with your life?

58:20

Look, my

58:23

prediction for Tencentorrent is that they're gonna pivot to making

58:25

RISC-V chips. CPUs.

58:29

CPUs. Why?

58:33

Because AI accelerators are

58:35

a software problem, not really a hardware problem. Oh,

58:38

interesting, so you don't think, you

58:40

think the diversity of AI accelerators

58:43

in the hardware space is not going to be

58:45

a thing that exists long-term.

58:47

I think what's gonna happen is if

58:49

I can, okay,

58:51

if you're trying to make an AI accelerator,

58:54

you better have the capability

58:56

of writing a torch-level

58:58

performance stack on NVIDIA GPUs.

59:01

If you can't write a torch stack on NVIDIA

59:03

GPUs, and I mean all the way, I mean down to the driver,

59:05

there's no way you're gonna be able to write it on your chip because

59:08

your chip's worse than an NVIDIA GPU. The first

59:10

version of the chip you tape out, it's definitely worse. Oh,

59:12

and you're saying writing that stack is really tough. Yes,

59:15

and not only that, actually, the chip that you tape out, almost

59:17

always because you're trying to get advantage over NVIDIA, you're

59:19

specializing the hardware more. It's

59:21

always harder to write software for more specialized

59:24

hardware. Like a GPU's pretty generic,

59:26

and if you can't write an NVIDIA stack, there's

59:28

no way you can write a stack for your chip. So

59:31

my approach with TinyGrad is first, write

59:33

a performant NVIDIA stack. We're targeting

59:36

AMD. So

59:38

you did say a few to NVIDIA a little bit, with

59:41

love. With love. Yeah, but so what- It's

59:43

like the Yankees, you know? I'm a Mets fan. Oh,

59:46

you're a Mets fan. A risk

59:48

fan and a Mets fan. What's the hope that AMD

59:51

has? You did a build

59:53

with AMD recently that I saw. That

59:56

was the PTSD. There,

59:59

the 79,

59:59

700 XTX compared to the

1:00:02

RTX 4090 or 4080. Well,

1:00:04

let's start with the fact that the 7900 XTX kernel

1:00:07

drivers don't work. And if you run demo

1:00:09

apps in loops, it panics the kernel.

1:00:11

Okay, so this is a softer issue.

1:00:15

Lisa Sue responded to my email.

1:00:17

Oh. I reached out, I was like,

1:00:19

this is, you know, really?

1:00:22

Like, I understand if your seven

1:00:25

by seven transposed Winograd comm

1:00:27

is slower than Nvidia's, but literally when

1:00:29

I run demo apps in a loop, the

1:00:31

kernel panics.

1:00:33

So just adding that loop.

1:00:36

Yeah, I just literally took their demo apps and wrote like, while

1:00:38

true, semi-colon do the app, semi-colon

1:00:41

done in a bunch of screens. Right,

1:00:43

this is like the most primitive fuzz testing.

1:00:46

Why do you think that is? They're just

1:00:48

not seeing a market in

1:00:51

machine learning? They're changing,

1:00:53

they're trying to change. They're trying to change.

1:00:55

And I had a pretty positive interaction with them this week.

1:00:57

Last week I went on YouTube, I was just like, that's

1:00:59

it. I give up on AMD. Like, this is their

1:01:02

driver,

1:01:02

doesn't even, I'm not gonna, I'll

1:01:05

go with Intel GPUs. Intel GPUs have

1:01:07

better drivers. So

1:01:10

you're kind of spearheading the

1:01:13

diversification of GPUs.

1:01:16

Yeah, and I'd like to extend that diversification

1:01:18

to everything. I'd like to diversify

1:01:20

the, right, the more,

1:01:25

my central thesis about the world is,

1:01:28

there's things that centralize power and they're bad. And

1:01:30

there's things that decentralize power and they're good.

1:01:33

Everything I can do to help decentralize power,

1:01:35

I'd like to do.

1:01:38

So you're really worried about the centralization of Nvidia, that's interesting.

1:01:41

And you don't have a fundamental hope for the

1:01:44

proliferation of ASICs, except

1:01:46

in the cloud.

1:01:49

I'd like to help them with software. No, actually, there's

1:01:51

only, the only ASIC that is remotely successful

1:01:54

is Google's TPU. And the only

1:01:56

reason that's successful is because Google wrote

1:01:58

a

1:01:59

machine learning framework.

1:02:00

I think that you have to write a competitive

1:02:02

machine learning framework in order to be able

1:02:04

to build an ASIC.

1:02:07

You think Meta with PyTorch builds

1:02:09

a competitor? I hope so. They

1:02:12

have one. They have an internal one. Internal.

1:02:14

I mean, public facing with a nice cloud

1:02:16

interface and so on. I don't want

1:02:18

a cloud. You don't like cloud. I don't

1:02:20

like cloud. What do you think is the fundamental

1:02:22

limitation of cloud? Fundamental limitation

1:02:25

of cloud is who owns the off switch.

1:02:27

So it's a power to the people. Yeah.

1:02:30

And you don't like the man to have all the

1:02:32

power. Exactly. All right.

1:02:35

And right now, the only way to do that is with AMD

1:02:37

GPUs if you want performance and

1:02:39

stability. Interesting.

1:02:43

But it's a costly investment emotionally

1:02:45

to go with AMDs. Well,

1:02:48

let me add sort of on a tangent to ask you, what did,

1:02:52

you've built quite a few PCs. What's your advice

1:02:54

on how to build a good custom PC for

1:02:57

let's say for the different applications they use for

1:02:59

gaming, for machine learning? Well, you

1:03:01

shouldn't build one. You should buy a box from the tiny Corp.

1:03:04

I heard rumors, whispers

1:03:08

about this box in the tiny Corp.

1:03:10

What's this thing look like? What is it?

1:03:12

What is it called? It's called the tiny box. Tiny

1:03:14

box? It's $15,000. And

1:03:18

it's almost a pit of flop of compute. It's

1:03:21

over a hundred gigabytes of GPU RAM. It's

1:03:23

over five terabytes per second of

1:03:26

GPU memory bandwidth.

1:03:29

I'm gonna put like four NVMe's in

1:03:31

RAID. You're gonna get

1:03:34

like 20, 30 gigabytes per second of drive read bandwidth.

1:03:38

I'm gonna build like the best

1:03:40

deep learning box that I can that

1:03:42

plugs into one wall outlet.

1:03:45

Okay. Can you go through those specs again a little

1:03:47

bit from memory? Yeah,

1:03:49

so it's almost a pit of flop of compute. So in

1:03:51

D and tell? Today I'm

1:03:53

leaning toward AMD, but

1:03:56

we're pretty agnostic to the type of compute.

1:03:59

The main limiting spec is a 120 volt 15

1:04:02

amp circuit.

1:04:06

Okay. Well, I mean it, because in order to like,

1:04:09

there's a plug over there, right?

1:04:12

You have to be able to plug it in.

1:04:14

We're also gonna sell the tiny rack, which

1:04:17

like, what's the most power you can

1:04:19

get into your house without arousing suspicion? And

1:04:22

one of the answers is an electric car

1:04:24

charger.

1:04:25

Wait, where does the rack go? Your

1:04:27

garage. Interesting. The

1:04:30

car charger.

1:04:31

A wall outlet is about 1500 watts. A

1:04:34

car charger is about 10,000 watts.

1:04:36

What is the most amount

1:04:38

of power you can get your hands on

1:04:40

without arousing suspicion? That's right.

1:04:42

George Haas. Okay. So

1:04:46

the tiny box and you said NVMe's and RAID.

1:04:49

I forget what you said about memory, all that kind of

1:04:51

stuff. Okay. What about with

1:04:53

GPUs? Again, probably 7900 XTX's,

1:04:58

but maybe 3090's, maybe A770's.

1:05:01

Those are intense. You're flexible

1:05:03

or still exploring? I'm still

1:05:05

exploring. I wanna deliver

1:05:07

a really good experience to people. And

1:05:11

yeah, what GPUs I end up going with. Again, I'm

1:05:13

leaning toward AMD.

1:05:14

We'll see. You know, in my

1:05:16

email, what I said to AMD is like,

1:05:19

just dumping the code on GitHub is not open

1:05:21

source.

1:05:22

Open source is a culture. Open

1:05:24

source means that your issues are not all

1:05:27

one year old stale issues. Open

1:05:29

source means developing

1:05:31

in public. And if you guys can commit

1:05:33

to that, I see a real future for

1:05:35

AMD as a competitor to Nvidia. Well,

1:05:39

I'd love to get a tiny box at MIT.

1:05:41

So whenever it's ready, let's

1:05:43

do it. We're taking pre-orders. I took this from Elon.

1:05:46

I'm like, all right, $100 fully refundable

1:05:48

pre-orders. Is it gonna be like the Cybertruck?

1:05:50

It's gonna take a few years or? No, I'll

1:05:52

try to do it faster. It's a lot simpler. It's a lot simpler

1:05:54

than the truck. Well, there's complexities

1:05:57

not to just the putting

1:05:59

the thing together.

1:05:59

about shipping and all this kind of stuff. The thing

1:06:02

that I want to deliver to people out of the box is

1:06:04

being able to run 65 billion parameter

1:06:06

llama in FP16

1:06:08

in real time, in a good, like 10 tokens

1:06:10

per second or five tokens per second or something. Just,

1:06:12

it works. Yep, just works. Llama's

1:06:15

running or something

1:06:17

like llama. Experience, yeah,

1:06:19

or I think Falcon is the new one, experience

1:06:22

a chat with the largest language model that

1:06:24

you can have in your house.

1:06:26

Yeah, from a wall plug. From

1:06:28

a wall plug, yeah. Actually, for inference,

1:06:31

it's not like even more power would help you get more.

1:06:34

Even more power wouldn't get you more. Well,

1:06:37

no, there's just the biggest, the biggest model released is 65

1:06:39

billion parameter llama as far as I know.

1:06:42

So it sounds like Tiny Box will naturally pivot

1:06:44

towards company number three, because you could just

1:06:46

get the girlfriend and,

1:06:50

or boyfriend. That

1:06:52

one's harder, actually. The boyfriend is harder? Boyfriend's

1:06:54

harder, yeah. I think that's a

1:06:56

very biased statement. I think

1:06:58

a lot of people would just say, why

1:07:01

is it harder to replace

1:07:03

a boyfriend than a other girlfriend

1:07:05

with the artificial llm? Because women

1:07:07

are attracted to status and power and men are

1:07:09

attracted to youth and beauty. No,

1:07:13

I mean, that's what I mean. Both are

1:07:15

a mimicable easy through the language model.

1:07:17

No, no machines do not

1:07:19

have any status or real power.

1:07:21

I don't know, I think you both, well,

1:07:24

first of all, you're using language mostly to

1:07:29

communicate youth

1:07:31

and beauty and power and status. But

1:07:33

status fundamentally is a zero sum game,

1:07:35

whereas youth and beauty are not.

1:07:37

No, I think status is a narrative you can construct.

1:07:40

I don't think status is real. I

1:07:44

don't know. I just think that that's why it's harder.

1:07:47

You know, yeah, maybe it is my biases. I

1:07:49

think status is way easier to fake. I

1:07:51

also think that, you know, men are probably

1:07:53

more desperate and more likely to buy my products, so

1:07:55

maybe they're a better target market. Desperation

1:07:58

is interesting, easier to fool.

1:07:59

Cool. Yeah. I could

1:08:02

see that. Yeah, look, I mean, look, I know you can look at porn

1:08:04

viewership numbers, right?

1:08:05

A lot more men watch porn than women.

1:08:07

You can ask why that is. Wow,

1:08:09

there's a lot of questions and answers

1:08:12

you can get there. Anyway,

1:08:15

with the TinyBox, how

1:08:17

many GPUs in TinyBox? Six.

1:08:20

Ha ha ha ha ha ha ha ha. Oh

1:08:24

man. And I'll tell you why it's six. Yeah. So

1:08:27

AMD Epic processors have 128 lanes of PCIe.

1:08:31

I want to leave enough lanes

1:08:34

for

1:08:35

some drives.

1:08:38

And I want to leave enough lanes for some networking.

1:08:41

How do you do cooling for something like this? Ah,

1:08:44

that's one of the big challenges. Not only

1:08:46

do I want the cooling to be good, I want it to be quiet.

1:08:48

I want the TinyBox to be able to sit comfortably

1:08:50

in your room, right? This is really going towards

1:08:53

the girlfriend thing. So,

1:08:55

because you want to run the LOM. I'll

1:08:57

give a more, I mean, I can talk about how it relates

1:08:59

to company number one.

1:09:01

Common AI. Yeah. Well,

1:09:05

but yes, quiet. Oh, quiet because you may

1:09:07

be potentially want to run it in a car. No, no

1:09:09

quiet because you want to put this thing in your house and you

1:09:11

want it to coexist with you. If it's screaming at 60 dB,

1:09:14

you don't want that in your house, you'll kick it out. 60 dB, yeah. I

1:09:17

want like 40, 45. So how do you make the cooling

1:09:20

quiet? That's an interesting problem in itself.

1:09:22

A key trick is to actually make it big. Ironically,

1:09:25

it's called the TinyBox. But if I can make

1:09:27

it big, a lot of that noise is generated

1:09:29

because of high pressure. If

1:09:31

you look at like a 1U server, a

1:09:33

1U server has these super high pressure fans that

1:09:35

are like super deep and they're like Genesis. Versus

1:09:38

if you have something that's big,

1:09:40

well, I can use a big thing. You know, they call

1:09:42

them big ass fans. Those ones that are like huge on

1:09:44

the ceiling and they're completely

1:09:46

silent. So TinyBox

1:09:48

will be big. I

1:09:52

do not want it to be large according to UPS.

1:09:54

I want it to be shippable as a normal package, but that's my

1:09:57

constraint there.

1:09:58

Interesting. The fans stuff,

1:10:01

can't it be assembled on location or no? No.

1:10:04

No, it has to be, wow. You're... Look,

1:10:07

I wanna give you a great out of the box experience. I want you to lift

1:10:09

this thing out. I want it to be like the Mac, you know?

1:10:12

Tiny box. The Apple experience.

1:10:14

Yeah. I love it. Okay,

1:10:17

and so Tiny Box would run

1:10:20

Tiny Grad. Like what

1:10:23

do you envision this whole thing to look like? We're

1:10:25

talking about like

1:10:26

Linux with the full...

1:10:30

Software engineering environment.

1:10:33

And it's just not PyTorch but Tiny Grad.

1:10:36

Yeah, we did a poll of people want you Bunto or

1:10:38

Arch. We're gonna stick with you Bunto. Ooh,

1:10:40

interesting. What's your favorite flavor

1:10:43

of Linux? You Bunto. Bunto. I

1:10:45

like you Bunto Mate, however you pronounce that,

1:10:47

meat. So

1:10:49

how do you, you've gotten llama into

1:10:52

Tiny Grad. You've gotten stable diffusion into

1:10:54

Tiny Grad. What was that like? Can you comment on like,

1:10:59

what are these models? What's interesting about porting

1:11:01

them? So what's, yeah,

1:11:03

like what are the challenges? What's

1:11:05

naturally, what's easy, all that kind of stuff. There's a

1:11:08

really simple way to get these models into Tiny

1:11:10

Grad and you can just export them as Onyx and

1:11:12

then Tiny Grad can run Onyx. So

1:11:15

the ports that I did of llama,

1:11:17

stable diffusion and now whisper

1:11:18

are more academic to teach me about

1:11:21

the models, but they are cleaner

1:11:23

than the PyTorch versions. You can read the code.

1:11:25

I think the code is easier to read. It's less lines.

1:11:28

There's just a few things about the way Tiny Grad writes

1:11:30

things. Here's a complaint I have about PyTorch. NN.relu

1:11:34

is a class,

1:11:36

right? So when you create an NN module, you'll

1:11:38

put your NN relu

1:11:41

as in a knit. And this makes

1:11:43

no sense. Relu is completely stateless.

1:11:46

Why should that be a class?

1:11:48

But that's more like a software engineering

1:11:51

thing. Or do you think it has a cost on performance?

1:11:53

Oh no, it doesn't have a cost on performance. But

1:11:56

yeah, no, I think that it's, that's

1:11:58

what I mean about Tiny Grad's front.

1:11:59

and being cleaner. I see.

1:12:03

What do you think about Mojo? I don't know if you've been paying attention

1:12:05

to the programming language that does some

1:12:08

interesting ideas that kind of intersect

1:12:10

TinyGrad.

1:12:11

I think that there is a spectrum. And

1:12:14

like on one side you have Mojo and on the other

1:12:16

side you have like GGML. GGML

1:12:19

is this like, we're gonna run llama fast on

1:12:21

Mac. Okay, we're gonna expand out

1:12:23

to a little bit, but we're gonna basically go like depth first,

1:12:26

right? Mojo is like, we're gonna go breath

1:12:28

first. We're gonna go so wide that we're gonna make all

1:12:30

of Python fast and TinyGrad's in the middle.

1:12:33

TinyGrad is, we are going

1:12:35

to make neural networks fast.

1:12:38

Yeah, but they try to really

1:12:41

get it to be fast, compiled

1:12:43

down to the specifics hardware

1:12:46

and make that compilation step

1:12:49

as flexible and resilient as possible.

1:12:51

Yeah, but they have Turing completeness.

1:12:53

And that limits you.

1:12:55

Turing. That's what you're saying, it's somewhere in

1:12:57

the middle. So you're actually going to be targeting some

1:12:59

accelerators, some number,

1:13:02

not one. My

1:13:05

goal is step one, build

1:13:07

an equally performance stack to PyTorch

1:13:09

on NVIDIA and AMD, but

1:13:12

with way less lines.

1:13:13

And then step two is, okay, how do

1:13:15

we make an accelerator? But you need step

1:13:17

one. You have to first build the framework before

1:13:20

you can build the accelerator. Can you explain

1:13:22

MLperf? What's your

1:13:24

approach in general to benchmarking TinyGrad performance?

1:13:27

So I'm much more

1:13:30

of a, like,

1:13:32

build it the right way and worry

1:13:34

about performance later.

1:13:36

There's a bunch of things where I haven't

1:13:38

even like,

1:13:39

really dove into performance. The only place

1:13:41

where TinyGrad is competitive performance wise right now

1:13:44

is on Qualcomm GPUs. So

1:13:46

TinyGrad's actually used an open pilot to run the model. So

1:13:49

the driving model is TinyGrad. When

1:13:51

did that happen? That transition?

1:13:53

About eight months ago now.

1:13:56

And it's 2x faster than Qualcomm's library. What's

1:13:59

the hardwood? where

1:14:01

that open pilot runs on the Kamaia.

1:14:04

It's a Snapdragon 845. Okay.

1:14:07

So this is using the GPU. So the GPU's an Adreno GPU.

1:14:10

There's like different things. There's a really good Microsoft

1:14:13

paper that talks about like mobile GPUs

1:14:15

and why they're different from desktop GPUs. One

1:14:18

of the big things is in a desktop

1:14:20

GPU, you can use buffers

1:14:22

on a mobile GPU image textures

1:14:24

a lot faster.

1:14:27

And a mobile GPU image textures, okay. And

1:14:30

so you want to be able to leverage

1:14:33

that.

1:14:34

I want to be able to leverage it in a way that it's completely

1:14:36

generic, right? So there's a lot of this. Xiaomi

1:14:38

has a pretty good open source library for

1:14:40

mobile GPUs called MACE, where

1:14:42

they can generate where they have these kernels,

1:14:45

but they're all hand coded, right? So

1:14:47

that's great if you're doing three by three comps. That's

1:14:49

great if you're doing dense map models. But the minute

1:14:51

you go off the beaten path a tiny bit, well, your

1:14:54

performance

1:14:54

is nothing. Since

1:14:56

you mentioned OpenPilot, I'd love to get an update

1:14:58

in the company number

1:15:01

one, Calm AI World. How are

1:15:03

things going there in the development of

1:15:07

semi-autonomous driving?

1:15:10

You know, almost no one talks

1:15:13

about FSD anymore, and even less people

1:15:15

talk about OpenPilot. We've solved

1:15:17

the problem. Like, we solved it years ago.

1:15:21

What's the problem exactly? Well,

1:15:23

how do you... What does solving it mean?

1:15:26

Solving means how do you build a model that

1:15:29

outputs a human policy for driving?

1:15:31

How do you build a model that, given a reasonable

1:15:34

set of sensors, outputs a human policy for driving?

1:15:37

So you have companies

1:15:39

like Waymo and Cruise, which are hand coding these things that

1:15:41

are like quasi human policies. Then

1:15:45

you have Tesla

1:15:47

and maybe even to more of an extent,

1:15:50

comma, asking, okay, how do we just learn the human

1:15:52

policy and data? The

1:15:55

big thing that we're doing now, and we just put it out on Twitter...

1:16:00

At the beginning of comma,

1:16:02

we published a paper

1:16:03

called Learning a Driving Simulator.

1:16:06

And the way this thing worked was it

1:16:09

was an auto encoder and

1:16:11

then an RNN in the middle. You

1:16:14

take an auto encoder, you compress

1:16:16

the picture,

1:16:17

you use an RNN, predict the next state, and

1:16:19

these things were. It was a laughably

1:16:22

bad simulator. This is

1:16:24

2015-hour machine learning technology. Today

1:16:26

we have VQ, VAE, and transformers.

1:16:29

We're building Drive GPT basically.

1:16:32

Drive GPT. It's

1:16:37

trained on what? Is it trained in a self-supervised

1:16:39

way? It's trained on all the driving data

1:16:41

to predict the next frame.

1:16:43

Really trying to learn

1:16:45

a human policy. What would a human do? Well,

1:16:48

actually our simulator is conditioned on the pose. It's

1:16:50

actually a simulator. You can put in a state action

1:16:52

pair and get out the next state.

1:16:54

And then once

1:16:56

you have a simulator, you can do RL

1:16:58

in the simulator and RL will get us that

1:17:01

human policy.

1:17:02

So transfers. Yeah.

1:17:05

RL with a reward function, not

1:17:07

asking is this close to the human policy, but asking

1:17:09

would a human disengage if you did this behavior?

1:17:12

Okay. Let me think about the distinction

1:17:15

there. What a human disengage. What

1:17:18

a human disengage. That

1:17:22

correlates, I guess, with human policy,

1:17:24

but it could be different. So

1:17:27

it doesn't just say what would a human

1:17:29

do. It says what

1:17:30

would a good human driver do and

1:17:32

such that the experience is comfortable,

1:17:36

but also not annoying in that the thing

1:17:38

is very cautious. So it's

1:17:41

finding a nice balance. That's interesting. It's

1:17:43

a nice... It's asking exactly the right question. What

1:17:46

will make our customers happy? Right.

1:17:49

A system that you never wanted to engage.

1:17:51

Because usually disengagement is almost

1:17:54

always a sign of I'm not

1:17:56

happy with what the system is doing. Usually.

1:17:59

There's some that are just I fell. like driving and those

1:18:01

are always fine too but they're just going to look like noise

1:18:03

in the data. But even

1:18:05

that felt like driving. Maybe

1:18:07

yeah. That's even that's a signal like why

1:18:09

do you feel like driving here you

1:18:12

need to recalibrate

1:18:14

your relationship with the car. Okay

1:18:17

so what that that's really interesting. How

1:18:20

close are we just solving self-driving?

1:18:25

It's hard to say.

1:18:26

We haven't completely closed the loop yet

1:18:29

so we don't have anything built that truly looks like

1:18:31

that architecture yet. We have prototypes

1:18:34

and there's bugs. So we are

1:18:36

a

1:18:36

couple bug fixes away. Might take

1:18:39

a year might take 10. What's the

1:18:41

nature of the bugs? Are these

1:18:44

these major philosophical bugs logical

1:18:46

bugs? What kind of what kind of bugs are we talking about?

1:18:48

They're just like they're just like stupid bugs and like

1:18:50

also we might just need more scale. We

1:18:52

just massively expanded our compute

1:18:54

cluster at Kama. We

1:18:57

now have about two people worth of compute 40

1:18:59

beta flops.

1:19:00

Well people people

1:19:03

are different. Yeah 20 beta

1:19:05

flops. That's a person. It's just a unit right. Horses

1:19:08

are different too but we still call it a horsepower. Yeah

1:19:11

but there's something different about mobility

1:19:13

than there is about

1:19:14

perception and action

1:19:17

in a very complicated world. But yes.

1:19:19

Well yeah of course not all flops are created equal. If you

1:19:21

have randomly initialized weights it's not gonna. Not

1:19:24

all flops are created equal. So flops

1:19:26

are doing way more useful things than others. Yeah.

1:19:31

Tell me about it. Okay so more

1:19:33

data scale means more scale in compute

1:19:35

or scale in scale of data?

1:19:37

Both. Diversity

1:19:41

of data? Diversity is very important in data.

1:19:43

Yeah I mean

1:19:45

we have so we have about I think

1:19:47

we have like 5 000 daily actives.

1:19:51

How would you evaluate how uh FSD

1:19:54

is doing? Pretty well.

1:19:56

How's that race going

1:19:58

between Kama AI and FSD?

1:19:59

Tesla has always wanted two years ahead of us. They've

1:20:02

always been wanted two years ahead of us. And they

1:20:04

probably always will be because they're not doing anything wrong. What

1:20:07

have you seen that's since the last time we talked that

1:20:09

are interesting architectural decisions, training decisions,

1:20:12

like the way the way they deploy stuff, the architectures

1:20:14

they're using in terms of the software,

1:20:16

how the teams are run, all that kind of stuff, data collection.

1:20:19

Anything interesting? I mean, I know they're moving

1:20:21

toward more of an end to end approach.

1:20:23

So creeping towards end to end

1:20:25

as much as possible across the

1:20:28

whole thing. The training, the data

1:20:30

collection, everything. They also have a very fancy simulator.

1:20:32

They're probably saying all the same things we are. They're

1:20:34

probably saying we just need to optimize, you know, what

1:20:37

is the reward? We get negative reward for this engagement.

1:20:39

Right? Like, everyone kind of knows this.

1:20:41

It's just a question who can actually build and deploy the system.

1:20:44

Yeah. I mean, this good, it's requires

1:20:46

good software engineering, I think. Yeah. And

1:20:49

the right kind of hardware. Yeah,

1:20:51

the hardware to run it. You

1:20:53

still don't believe in cloud in that regard?

1:20:57

I have a compute cluster

1:20:59

in my office. 800 amps. Tiny

1:21:02

grad. It's 40 kilowatts at idle,

1:21:04

our data center.

1:21:06

That's crazy. If 40 kilowatts is burning

1:21:08

just when the computers are idle.

1:21:09

Just when I... Oh, sorry. Sorry. Compute cluster. Compute

1:21:14

cluster. I got it. It's not a data center. Yeah. Now,

1:21:16

data centers are clouds. We don't have clouds.

1:21:19

Data centers have air conditioners. We have fans. That

1:21:22

makes it a compute cluster. I'm

1:21:25

guessing this is a kind of a legal distinction

1:21:27

as compared to me. Sure. Yeah. We have a compute

1:21:29

cluster.

1:21:31

You said that you don't think LLMs have consciousness,

1:21:33

or at least not more than a chicken.

1:21:36

Do you think they can reason? Is there something

1:21:38

interesting to you about the word reason, about

1:21:41

some of the capabilities that we think is kind of human,

1:21:43

to be able to

1:21:45

integrate

1:21:47

complicated information and through

1:21:50

a chain of thought arrive

1:21:54

at a conclusion that feels novel, a novel

1:21:57

integration of the... disparate

1:22:00

facts. Yeah,

1:22:03

I don't think that there's, I think that

1:22:05

they can reason better than a lot of people. Hey,

1:22:08

isn't that amazing to you though? Isn't

1:22:10

that like an incredible thing that a transformer

1:22:12

can achieve? I mean, I think

1:22:14

that calculators can add better than a lot

1:22:16

of people. But language feels

1:22:19

like reasoning through the process

1:22:21

of language, which

1:22:23

looks a lot like thought. Making

1:22:27

brilliant season chess, which feels

1:22:29

a lot like thought. Whatever new

1:22:31

thing that AI can do, everybody thinks is

1:22:33

brilliant. And then like 20 years go by and they're like,

1:22:35

well, you have a chess, that's like mechanical. Like adding,

1:22:37

that's like mechanical. So you think language is not

1:22:39

that special. It's like chess. It's like chess

1:22:42

and it's like- I don't know. Because it's very

1:22:44

human, we take it, listen,

1:22:47

there's something different between chess and

1:22:51

language. Chess is a game that a subset

1:22:53

of the population plays. Language is something

1:22:56

we

1:22:56

use nonstop for all

1:22:58

of our human interaction. And human interaction

1:23:01

is fundamental to society. So

1:23:03

it's like, holy shit. This

1:23:06

language thing is not so difficult to

1:23:08

like

1:23:09

create in the machine. The problem

1:23:12

is if you go back to 1960 and you

1:23:14

tell them that you have a machine that can play

1:23:17

amazing chess,

1:23:19

of course someone in 1960 will tell you that machine

1:23:21

is intelligent.

1:23:23

Someone in 2010 won't, what's changed,

1:23:25

right? Today, we think that these machines

1:23:27

that have language are intelligent. But

1:23:30

I think in 20 years, we're gonna be like, yeah, but can it

1:23:32

reproduce?

1:23:33

So reproduction, yeah,

1:23:36

we might redefine what it means to be,

1:23:39

what is it? A high performance living

1:23:41

organism on earth. Humans are always gonna

1:23:44

define a niche for themselves. Like, well,

1:23:46

you know, we're better than the machines because we can,

1:23:48

you know, and like they tried creative for a bit, but

1:23:50

no one believes that one anymore.

1:23:52

But niche, is

1:23:54

that delusional? There's some accuracy to that. Because

1:23:57

maybe like with chess, you start to realize like.

1:23:59

that

1:24:02

we have, it'll conceive notions of

1:24:05

what makes humans special.

1:24:07

Like the apex organism

1:24:09

on Earth. Yeah,

1:24:12

and I think maybe we're going to go through that same

1:24:14

thing with language.

1:24:16

And that same thing with creativity.

1:24:19

But

1:24:19

language carries these notions of truth

1:24:22

and so on. And so we might be like, wait,

1:24:24

maybe truth is not carried by language.

1:24:27

Maybe there's like a deeper thing. The niche is

1:24:29

getting smaller. Oh, boy. But

1:24:33

no, no, no, you don't understand humans are

1:24:35

created by God and machines are created

1:24:37

by humans, therefore. Right? Like that'll

1:24:39

be the last niche we have.

1:24:41

So what do you think about this,

1:24:43

the rapid development of LMS? If we could

1:24:45

just like stick on that. It's still incredibly

1:24:47

impressive, like with Chagibiti. Just even Chagibiti,

1:24:49

what are your thoughts about reinforcement

1:24:52

learning with human feedback on these large language

1:24:54

models?

1:24:55

I'd like to go back to when calculators

1:24:58

first came out

1:24:59

and or computers. And

1:25:02

like I wasn't around. Look, I'm 33 years old. And

1:25:05

to like

1:25:06

see how that affected

1:25:09

like

1:25:12

society.

1:25:13

Maybe you're right. I want to put on

1:25:16

the the big picture

1:25:18

hat here. Oh my God, a refrigerator? Wow.

1:25:21

Refrigerator, electricity, all that kind of stuff.

1:25:25

But no,

1:25:26

with the Internet,

1:25:28

large language models seeming human like

1:25:31

basically passing a Turing test. It

1:25:33

seems it might have really at scale

1:25:36

rapid transformative effects on society.

1:25:39

But you're saying like other technologies have as well.

1:25:43

So maybe calculators not the best

1:25:45

example that because that just seems

1:25:47

like a may. Well, no, maybe

1:25:49

calculator. The poor milk man, the day he

1:25:51

learned about refrigerators, he's like, I'm done. You

1:25:55

tell me you just keep the milk in your house. You

1:25:58

don't need to deliver it every day. I'm done.

1:26:00

Well, yeah, you have to actually look at the practical

1:26:02

impacts of certain technologies that they've

1:26:04

had. Yeah, probably electricity

1:26:07

is a big one and also how rapidly it spread.

1:26:10

Man, the internet is a big one. I do think it's different

1:26:12

this time though.

1:26:13

Yeah, it just feels like stuff- The initiative is getting

1:26:15

smaller. The initiative is

1:26:17

humans. Yes. That

1:26:20

makes humans special. Yes. It

1:26:22

feels like it's getting smaller rapidly though, doesn't

1:26:25

it? Or is that just a feeling we dramatize

1:26:27

everything? I think we dramatize everything. I

1:26:29

think that you asked the milk

1:26:32

man when he saw the refugee writers, and they're

1:26:34

going to have one of these in every home? Yeah,

1:26:38

yeah, yeah. Yeah,

1:26:41

but boy, is it impressive. So

1:26:44

much more impressive than seeing a

1:26:47

chess world champion AI system. I

1:26:49

disagree, actually.

1:26:51

I disagree. I

1:26:53

think things like Mu Zero and AlphaGo

1:26:55

are so much more impressive because

1:26:57

these things are playing beyond

1:27:00

the highest human level. The

1:27:03

language models are writing middle

1:27:06

school level essays, and people are like, wow,

1:27:08

it's a great essay. It's a great five-paragraph

1:27:10

essay about the causes of the Civil War. Okay,

1:27:13

forget the Civil War, just generating code, codex.

1:27:15

Oh. So you're saying it's

1:27:18

mediocre code. Terrible. But

1:27:20

I don't think it's terrible. I think it's just mediocre

1:27:23

code. Yeah.

1:27:25

Often

1:27:25

close to correct. Like

1:27:27

for mediocre purposes. That's the scariest kind

1:27:30

of code. I spend 5% of time typing and 95% of time debugging.

1:27:33

The last thing I want is close to correct code.

1:27:36

I want a machine that can help me with the debugging, not

1:27:38

with the typing. You know, it's like level

1:27:40

two driving, similar

1:27:42

kind of thing. Yeah, you still should

1:27:45

be a good programmer in order to modify.

1:27:48

I wouldn't even say debugging. It's just modifying

1:27:50

the code, reading it. Don't think it's like level

1:27:52

two driving.

1:27:54

I think driving is not tool complete and programming

1:27:56

is. Meaning you don't use like the best

1:27:58

possible tools to drive. You're

1:28:01

not like, like, like,

1:28:03

cars have basically the same interface for

1:28:05

the last 50 years. Computers have

1:28:07

a radically different interface. Okay. Can

1:28:09

you describe the concept of tool complete?

1:28:12

Yeah. So think about the difference between a car from 1980 and

1:28:15

a car from today. Yeah. No difference

1:28:17

really. It's got a bunch of pedals, it's got a steering wheel.

1:28:20

Great.

1:28:20

Maybe now it has a few ADAS features, but

1:28:23

it's pretty much the same car. All right. You

1:28:25

have no problem getting into a 1980 car and driving it. Take

1:28:28

a programmer today who spent their whole life doing JavaScript

1:28:31

and you put him in an Apple 2e prompt and

1:28:33

you tell him about the line numbers in basic.

1:28:36

But how do I insert

1:28:38

something between line 17 and 18? Oh,

1:28:41

wow.

1:28:42

But the,

1:28:45

so in tool you're putting in the programming

1:28:47

languages. So it's just the entirety stack

1:28:49

of the tooling. Exactly. So it's not just

1:28:51

like the IDs or something like this. It's everything.

1:28:54

Yes. It's IDEs, the languages, the runtimes.

1:28:56

It's everything. It's tool complete.

1:28:59

So like almost if, if,

1:29:01

if, if, if, if codex or

1:29:03

co-pilot are helping you, that

1:29:05

actually probably means that your framework or library is

1:29:08

bad and there's too much boilerplate in it.

1:29:12

Yeah. But don't you think

1:29:14

so much programming has boilerplate? Tinygrad

1:29:17

is now 2,700 lines

1:29:19

and it can run llama and stable

1:29:21

diffusion and all of this stuff

1:29:24

is in 2,700 lines. boilerplate

1:29:26

and abstraction

1:29:29

indirection and all these things are

1:29:31

just bad code. Well,

1:29:36

let's talk about good code and bad

1:29:38

code. There's a, I would

1:29:40

say, I don't know, for generic

1:29:42

scripts that I write just offhand, like

1:29:45

I, like 80% of it is written by GPT.

1:29:48

Just like quick, quick like offhand

1:29:50

stuff. So not like libraries, not like performing

1:29:53

code, not stuff for robotics and so

1:29:55

on. Just quick stuff because your basics,

1:29:57

so much of programming is doing some.

1:29:59

some, yeah, boilerplate, but

1:30:02

to do so efficiently and quickly,

1:30:06

because you can't really automate it fully with

1:30:08

like generic method, like a generic

1:30:11

kind of ID

1:30:13

type of recommendation or something like this,

1:30:15

you do need to have some of the complexity of

1:30:17

language models.

1:30:19

Yeah, I guess if I was really writing like

1:30:21

maybe today, if I wrote like a

1:30:23

lot of like data parsing stuff, I mean,

1:30:25

I don't play CTFs anymore, but if I still play CTFs,

1:30:27

a lot of the like, is just like you have to write like a parser for this

1:30:29

data format, like I wonder, or

1:30:32

like advent of code,

1:30:33

I wonder when the models are gonna

1:30:35

start to help with that kind of code,

1:30:37

and they may, they may, and the models

1:30:39

also may help you with speed, and the

1:30:41

models are very fast, but where

1:30:43

the models won't, my programming

1:30:46

speed is not at all limited by

1:30:48

my typing speed.

1:30:52

And in very few cases

1:30:54

it is, yes, if I'm writing some script to

1:30:56

just like parse some weird data format, sure,

1:30:59

my programming speed is limited by my typing speed. What about

1:31:01

looking stuff up? Because that's essentially

1:31:03

a more efficient look up, right? You know,

1:31:06

when I was at Twitter, I tried to use chat

1:31:09

GPT to like ask

1:31:11

some questions, like was the API for this? And

1:31:14

it would just hallucinate,

1:31:15

it would just give me completely made up API

1:31:18

functions that sounded real. Well,

1:31:20

do you think that's just a temporary kind of stage?

1:31:23

No. You don't think it'll

1:31:25

get better and better and better and this kind of stuff, because like

1:31:27

it only hallucinates stuff in the edge

1:31:29

cases. Yes.

1:31:30

If you're writing generic code, it's actually pretty good. Yes,

1:31:32

if you are writing an absolute basic like

1:31:34

React app with a button, it's not gonna hallucinate,

1:31:36

sure. No, there's kind of

1:31:38

ways to fix the hallucination problem. I think Facebook

1:31:41

has an interesting paper, it's called Atlas, and

1:31:43

it's actually weird the way that we do language

1:31:46

models right now where all of the

1:31:49

information is in the weights.

1:31:51

And human brains don't really like this. It's

1:31:53

like a hippocampus and a memory system. So

1:31:55

why don't LLMs have a memory system? And there's

1:31:57

people working on them. I think future LLMs are gonna.

1:31:59

be smaller, but

1:32:02

are going to run looping

1:32:04

on themselves and are going to have retrieval systems.

1:32:07

And the thing about using a retrieval system is you can

1:32:09

cite sources,

1:32:10

explicitly. Which

1:32:14

is really helpful to integrate

1:32:16

the human into the loop of the

1:32:18

thing, because you can go check the sources and you can investigate.

1:32:21

So whenever the thing is hallucinating, you can

1:32:23

have the human supervision. That's pushing

1:32:26

it towards level two kind of drive. That's gonna kill Google.

1:32:29

Wait, which part? When someone makes an LLM

1:32:31

that's capable of citing its sources, it will kill Google.

1:32:34

LLM that's citing its sources because that's basically

1:32:37

a search engine.

1:32:38

That's what people want in a search engine. But

1:32:40

also Google might be the people that build it. Maybe.

1:32:43

And put ads on it. I'd count them out. Why

1:32:46

is that? Why do you think? Who wins

1:32:48

this race? We got,

1:32:51

who are the competitors? We got

1:32:54

TinyCorp. I don't know if that's, yeah,

1:32:57

I mean, you're a legitimate competitor in that.

1:32:59

I'm not trying to compete on that. You're not.

1:33:02

No, not as a skit. It's gonna accidentally stumble into that

1:33:04

competition. Maybe. to

1:33:07

replace Google search?

1:33:08

When I started Comma, I said,

1:33:11

over and over again, I'm going to win self-driving cars.

1:33:13

I still believe that.

1:33:15

I have never said I'm going to win search

1:33:17

with the TinyCorp and

1:33:19

I'm never going to say that because I won't. The night

1:33:21

is still young. You don't know how

1:33:23

hard is it to win search in

1:33:25

this new route. It

1:33:28

feels, I mean, one of the things that ChatGPT kind of shows

1:33:30

that there could be a few interesting tricks that

1:33:32

really have, that create a really compelling product. Some

1:33:35

startup's gonna figure it out. I think

1:33:37

if you ask me, like Google's still the number one

1:33:39

webpage, I think by the end of the decade, Google won't be

1:33:41

the number one webpage anymore.

1:33:43

So you don't think Google, because of

1:33:45

the, how big the corporation is?

1:33:47

Look, I would put a lot more money on Mark Zuckerberg.

1:33:50

Why is that? Because

1:33:53

Mark Zuckerberg's alive. Like

1:33:57

this is old Paul Graham essay. Startups are

1:33:59

either alive or dead. Google's dead.

1:34:02

Facebook's alive. Versus Facebook is alive,

1:34:04

Meta is alive. Meta. Meta. You

1:34:06

see what I mean? Like that's just, like Mark

1:34:08

Zuckerberg, this is Mark Zuckerberg reading that Paul Graham

1:34:10

asking and being like, I'm gonna show everyone how alive

1:34:12

we are. I'm gonna change the name.

1:34:14

So you don't think there's this gutsy

1:34:18

pivoting engine that,

1:34:22

like Google doesn't have that, the kind of engine

1:34:24

that a startup has like constantly being

1:34:27

alive, I guess. When I listened to your Sam

1:34:29

Altman

1:34:30

podcast, he talked about the button. Everyone

1:34:32

who talks about AI talks about the button, the button to turn it off,

1:34:34

right? Do we have a button to turn off Google?

1:34:37

Is anybody

1:34:39

in the world capable of shutting Google down? What

1:34:43

does that mean exactly? The company or the search

1:34:45

engine? Could we shut the search engine down? Could we shut the company

1:34:47

down? Either. Can

1:34:50

you elaborate on the value of that question? Does

1:34:52

Sundar Pichai have the authority to turn

1:34:54

off google.com tomorrow? Who

1:34:57

has the authority? That's a good question. Does

1:35:00

anyone? Does anyone? Yeah, I'm sure.

1:35:03

Are you sure?

1:35:04

No, they have the technical power, but do they

1:35:06

have the authority? Let's say Sundar Pichai made

1:35:09

this his sole mission. He came into Google

1:35:11

tomorrow and said, I'm gonna shut google.com down.

1:35:14

I don't think he'd keep his position too long. And

1:35:18

what is the mechanism by which he wouldn't keep his position?

1:35:21

Well, boards and shares

1:35:23

and corporate undermining and oh my

1:35:25

God, our revenue is zero now. Okay,

1:35:29

so what's the case you're making here? So the

1:35:31

capitalist machine prevents you from having

1:35:34

the button. Yeah,

1:35:35

and it will have, I mean, this is true for the AIs too. There's

1:35:38

no turning the AIs off.

1:35:40

There's no button. You can't press it. Now,

1:35:42

does Mark Zuckerberg have that button for Facebook.com?

1:35:46

Yes, probably more. I think he does. I

1:35:49

think he does, and this is exactly what I mean

1:35:51

and why I bet on him so much more than

1:35:53

I bet on Google. I guess you could say Elon

1:35:55

has similar stuff. Oh, Elon has the button.

1:35:59

Yeah.

1:36:00

Does he want can you on fire the missiles? Can

1:36:02

he fire the missiles? I

1:36:04

think some questions are better I'm

1:36:07

asked I mean,

1:36:09

you know a rocket an ICB. Yeah, well your rocket

1:36:11

that can land anywhere. Is that an ICB? M? Well,

1:36:14

yeah, you know don't ask too many questions my

1:36:17

god

1:36:19

But the

1:36:21

the positive side of the button is that you can

1:36:23

innovate aggressively is what you're saying

1:36:25

which is what's required with Turning

1:36:28

LLM into a search engine. I would bet on a startup.

1:36:31

I bet is it so easy, right? I bet on something that looks

1:36:33

like mid-journey but for search

1:36:37

Just is able to set sources loop

1:36:39

on itself, I mean just feels like one model can take off

1:36:41

Yeah, right and that nice wrapper and some

1:36:43

of it scale. I mean, it's hard to Like

1:36:46

create a product that just works really nicely Stably

1:36:49

the other thing that's gonna be cool is there is

1:36:51

some aspect of a winner take all effect, right?

1:36:54

Like once um

1:36:56

Someone starts deploying a product that gets a lot

1:36:58

of usage and you see this with open AI They

1:37:00

are going to get the data set

1:37:02

to train future versions of the model

1:37:04

Yeah, they are going to be able to right, you

1:37:07

know I was asked a Google image search when I worked there like

1:37:09

almost 15 years ago now How does Google know which

1:37:11

image is an apple

1:37:12

and I said the metadata and they're like, yeah that

1:37:14

works about half the time How does Google know

1:37:16

you'll see the role apples on the front page when you search Apple?

1:37:19

Mm-hmm. And I don't know.

1:37:21

I didn't come up with the answer

1:37:22

The guys like what's what people click on when they search Apple? Yeah,

1:37:26

yeah that data is really really powerful. It's the

1:37:28

human supervision What do you think

1:37:30

are the chances? What do you think in general

1:37:33

that llama was open-sourced? I just

1:37:36

did a conversation with With

1:37:38

Mark Zuckerberg and he's all

1:37:41

in on open source

1:37:43

Who would have thought that Mark Zuckerberg

1:37:45

would be the good guy? I Mean

1:37:47

it Would have thought anything

1:37:50

in this world It's hard to know

1:37:53

but open source to you ultimately

1:37:57

Is a good thing here undoubtedly

1:38:01

You know, what's ironic

1:38:03

about all these AI safety people is

1:38:05

they are going to build the exact thing they fear.

1:38:09

These we need to have one model that we

1:38:11

control and align. This is

1:38:13

the only way you end up paper clipped. There's

1:38:16

no way you end up paper clipped if everybody

1:38:18

has an AI. So open sourcing is

1:38:20

the way to fight the paper clip, Maximizer? Absolutely.

1:38:23

It's the only way. You think you're going to control

1:38:26

it? You're not going to control it. So the

1:38:28

criticism you have for the AI safety folks

1:38:31

is that there is a belief

1:38:33

and a desire for control. And

1:38:36

that belief and desire for centralized

1:38:38

control of dangerous AI systems

1:38:41

is not good. Sam Altman won't tell you

1:38:43

that GPT-4 has 220 billion

1:38:46

parameters and is a 16-way mixture model

1:38:48

with eight sets of weights.

1:38:50

Who did you have to murder to

1:38:52

get that information? All right. I

1:38:54

mean, look. But yes. Everyone

1:38:57

at OpenAI knows what I just said was true. Now

1:39:01

ask the question, really.

1:39:03

It upsets me when I like GPT-2.

1:39:06

When OpenAI came out with GPT-2 and raised a whole

1:39:08

fake AI safety thing about that, I mean, now the

1:39:10

model is laughable.

1:39:12

They used AI safety

1:39:14

to hype up their company and it's disgusting.

1:39:18

Or the flip side of that is

1:39:21

they used a relatively weak model

1:39:23

in retrospect to explore how

1:39:25

do we do AI safety correctly?

1:39:27

How do we release things? How do we go through the process? I

1:39:30

don't know if... Sure. Sure.

1:39:33

All right. All right. That's

1:39:35

the charitable interpretation. I don't know how much hype there is in AI safety,

1:39:37

honestly. Oh, there's so much. At least on Twitter.

1:39:40

I don't know. Maybe Twitter's not real life. Twitter's

1:39:42

not real life. Come on. In

1:39:44

terms of hype. I mean, I don't... I

1:39:47

think OpenAI has been finding an

1:39:49

interesting balance between transparency

1:39:50

and putting value on

1:39:53

AI safety. You

1:39:55

think just go all out open

1:39:57

source. So do a llama. Absolutely.

1:40:00

So do like open source, this

1:40:02

is a tough question, which is open

1:40:05

source, both the base, the

1:40:07

foundation model and the fine tune

1:40:09

one. So like the

1:40:11

model that can be ultra racist and dangerous

1:40:14

and like tell you how to build

1:40:16

a nuclear weapon. Oh my God, have you met humans,

1:40:19

right? Like half of these AI- I haven't

1:40:21

met most humans. This makes,

1:40:23

this allows you to meet every human.

1:40:26

Yeah, I know, but half of these AI alignment

1:40:28

problems are just human alignment problems. And

1:40:30

that's what's also so scary about the language they

1:40:32

use. It's like, it's not the machines you want to align,

1:40:34

it's me.

1:40:37

But here's the thing, it

1:40:39

makes it very accessible to ask

1:40:43

very questions where

1:40:46

the answers have dangerous consequences if

1:40:48

you were to act on them. I

1:40:51

mean, yeah, welcome to the world.

1:40:54

Well, no, for me, there's a lot of friction. If

1:40:56

I want to find out how to,

1:40:59

I don't know, blow

1:41:01

up something. No, there's not a lot of friction that's

1:41:03

so easy. No, like what do I search

1:41:06

that is Bing? Or do I search anything that

1:41:08

I use? No, there's like lots of stuff. No,

1:41:10

it feels like I have to keep clicking a lot of this. First off, first off,

1:41:13

first off, anyone who's stupid enough to search for how

1:41:15

to blow up a building in my neighborhood

1:41:18

is not smart enough to build a bomb, right?

1:41:20

Are you sure about that? Yes. I

1:41:24

feel like a language model makes it

1:41:26

more accessible for that

1:41:29

person who's not smart enough to do. They're

1:41:31

not gonna build a bomb, trust me. The

1:41:34

people who are incapable of figuring

1:41:36

out how to ask that question a bit more academically

1:41:39

and get a real answer from it are not capable

1:41:41

of procuring the materials, which are somewhat controlled

1:41:43

to build a bomb. No, I think

1:41:45

it all makes it more accessible to people with money

1:41:48

without the technical know-how,

1:41:50

right? Like, do you really

1:41:52

need to know how to build a bomb to build a bomb? You

1:41:54

can hire people, you can find like- Or you can hire

1:41:57

people to build a- You know what? I was asking this question

1:41:59

on my stream. Like, can Jeff-

1:41:59

Bezos hire a hit man probably not but

1:42:03

a language model can

1:42:05

probably help you out yeah you'll

1:42:07

still go to jail right like it's not like the language

1:42:10

model is God like the language model it's like it's

1:42:12

you literally just hired someone on Fiverr

1:42:15

but you use it but okay GPT

1:42:17

for in terms of finding hit man is like asking

1:42:19

Fiverr how to find a I understand but

1:42:21

don't you think you how you know okay how but

1:42:23

don't you think GPT 5 will be better just

1:42:26

don't you think that information is out there on the internet I

1:42:29

mean yeah

1:42:29

and I think that if someone is actually

1:42:31

serious enough to hire a hit man or build a bomb

1:42:34

they'd also be serious enough to find the information

1:42:36

I don't think so I think it makes it more accessible

1:42:38

if you have if you have enough money to buy

1:42:40

hit man I think it decreases

1:42:43

the friction of how hard is it to find

1:42:45

that kind of hit man I honestly think this

1:42:48

there's a jump in

1:42:51

ease and scale of how

1:42:53

much harm you can do and I don't mean harm with language

1:42:56

I mean harm with actual violence what you're basically

1:42:58

saying is like okay what's gonna happen is these people

1:43:00

who are not intelligent are going to use