【AI安全】AI安全专家：2030年仅存的5类工作 · 未来

共 2 集

剧集目录

P1 · 【AI安全】AI安全专家：2030年仅存的5类工作 · 未来职业预测 p01 part 1 (P1)

【AI安全】AI安全专家：2030年仅存的5类工作 · 未来职业预测 p01 part 1 (P1)

发布于 2026-08-13 · 时长 1:00:07

字幕摘录

时间	英文	中文
0:00	You've been working on AI safety for two decades at least.	你至少为AI安全工作了20年
0:02	Yeah, I was convinced we can make safe AI, but the more I looked at it, the more I realized	是的,我确信我们可以 安全AI,但越看越明白
0:07	it's not something we can actually do.	这不是我们真正可以做的。
0:09	You have made a series of predictions about a variety of different dates.	你们对各种不同的日期作了一系列的预测。
0:13	So what is your prediction for 2027?	那你对2027年的预测是什么?
0:19	Dr. Roman Yampolsky is a globally recognized voice on AI safety and associate professor	Roman Yampolsky博士是AI安全方面的全球知名声音和副教授
0:24	of computer science.	计算机科学
0:26	He educates people on the terrifying truth of AI and what we need to do to save humanity.	他教育人们了解AI的可怕真相,以及我们需要做什么来拯救人类.
0:30	In two years, the capability to replace most humans and most occupations will come very	两年后,取代大多数人类和大多数职业的能力将非常高
0:35	quickly.	快点

展开字幕全文（1232 条）

序号	英文	中文
1	You've been working on AI safety for two decades at least.	你至少为AI安全工作了20年
2	Yeah, I was convinced we can make safe AI, but the more I looked at it, the more I realized	是的,我确信我们可以 安全AI,但越看越明白
3	it's not something we can actually do.	这不是我们真正可以做的。
4	You have made a series of predictions about a variety of different dates.	你们对各种不同的日期作了一系列的预测。
5	So what is your prediction for 2027?	那你对2027年的预测是什么?
6	Dr. Roman Yampolsky is a globally recognized voice on AI safety and associate professor	Roman Yampolsky博士是AI安全方面的全球知名声音和副教授
7	of computer science.	计算机科学
8	He educates people on the terrifying truth of AI and what we need to do to save humanity.	他教育人们了解AI的可怕真相,以及我们需要做什么来拯救人类.
9	In two years, the capability to replace most humans and most occupations will come very	两年后,取代大多数人类和大多数职业的能力将非常高
10	quickly.	快点
11	And then in five years, we're looking at a world where we have levels of unemployment	然后在五年内,我们面对一个 失业率高的世界
12	we've never seen before.	我们从来没有见过。
13	Not talking about 10%, but 99%.	不是说10%,而是99%。
14	And that's without super intelligence, a system smarter than all humans in all domains.	这是没有超级智能的, 一个比所有领域的人类都聪明的系统。
15	So it would be better than us at making new AI.	所以比我们做新的人工智能要好
16	But it's worse than that.	然则更恶.
17	We don't know how to make them safe.	我们不知道怎么让他们安全
18	And yet we still have the smartest people in the world competing to win the race to	但我们还有世界上最聪明的人 竞相赢得比赛
19	But what do you make of people like Sam Opland's journey with AI?	但你对萨姆·奥普兰和AI的旅程这样的人有什么看法?
20	So decades ago, we published cardrails for how to do AI right.	几十年前,我们出版了 如何做AI正确。
21	They violated every single one.	他们侵犯每一个。
22	And he's gambling 8 billion lives on getting richer and more powerful.	他赌80亿 生活越来越富强。
23	So I guess some people want to go to Mars, others want to control the universe.	我想有些人想去火星,有些人想控制宇宙。
24	But it doesn't matter who builds it.	但谁造之则无妨.
25	The moment you switch to super intelligence, we will most likely regret it terribly.	一旦你转向超级智能 我们很可能会非常后悔
26	And then by 2045.	然后到2045年为止。
27	Now this is where it gets interesting.	现在这里变得有趣了。
28	Dr. Roman Yimpolski.	罗曼・依姆波斯基博士
29	Let's talk about simulation theory.	让我们谈谈模拟理论。
30	I think we are in one.	我认为我们在一个。
31	And there is a lot of agreement on this.	这一点有很多共识。
32	And this is what you should be doing in it, so we don't shut it down.	这就是你应该做的, 所以我们不关闭它。
33	First.	先说.
34	I see messages all the time in the comments section that some of you didn't realize you	我经常在评论栏看到留言 说你们有些人没意识到你
35	didn't subscribe.	没有订阅。
36	So if you could do me a favor and double check if you're a subscriber to this channel,	所以如果你能帮我一个忙 再检查一下你是否是这个频道的订户
37	that would be tremendously appreciated.	这将是非常感激的。
38	It's the simple, it's the free thing that anybody that watches the show frequently can	这是简单的,这是自由的东西,任何人 谁经常看节目可以
39	do to help us here to keep everything going in this show, in the trajectory it's on.	做帮助我们在这里 保持一切在这场演出, 它的轨迹。
40	So please do double check if you subscribed.	请检查您是否订阅 。
41	And thank you so much, because in a strange way, you are you're part of our history and	非常感谢,因为 奇怪的是 你是我们历史的一部分
42	you're on this journey with us.	你和我们一起在旅途中
43	And I appreciate you for that.	我很感激你这样做。
44	So thank you.	谢谢
45	Roman Yimpolski博士 你现在的任务是什么?
46	Because it's quite clear to me that you are on a bit of a mission and you've been on	因为我很清楚你正在执行任务 而你一直在执行任务
47	this mission for I think the best part of two decades at least.	这个任务,我认为至少20年中最好的部分。
48	I'm hoping to make sure that super intelligence we're creating right now does not kill everyone.	我希望确保我们现在创造的超级情报不会杀光所有人.
49	Give me some give me some context on that statement, because it's quite a shocking statement.	给我一些关于那个声明的背景, 因为这是一个相当令人震惊的声明。
50	Sure.	当然
51	So the last decade, we actually figured out how to make artificial intelligence better.	因此,在过去十年里,我们实际上找到了 如何使人工智能更好。
52	Turns out if you add more compute, more data, it just kind of becomes smarter.	如果你增加更多的计算,更多的数据, 它就会变得更聪明。
53	And so now smartest people in the world, billions of dollars, all going to create the best possible	所以现在世界上最聪明的人, 数十亿美元, 都将创造出最好的
54	super intelligence we can.	超级智能,我们可以。
55	Unfortunately, while we know how to make those systems much more capable, we don't know how	不幸的是,虽然我们知道如何使这些系统更有能力,但我们不知道如何
56	to make them safe, how to make sure they don't do something we will regret.	让他们安全 如何确保他们不做我们后悔的事
57	And that's the state of the art right now.	这就是现在的艺术状态。
58	When we look at just prediction markets, how soon will we get to advanced AI?	当我们仅仅看看预测市场时,我们什么时候才能到达高级AI?
59	The timelines are very short, a couple of years, two, three years, according to prediction	根据预测,时间很短,几年,两年,三年
60	markets, according to CEOs of top labs.	根据顶级实验室的首席执行官
61	And at the same time, we don't know how to make sure that the systems are aligned with	同时,我们不知道如何确保系统与
62	our preferences.	我们的喜好。
63	So we're creating this alien intelligence.	所以我们正在创造这个外星人的情报。
64	If aliens were coming to Earth and you had three years to prepare, you would be panicking	如果外星人来到地球,你还有三年时间准备, 你会惊慌失措
65	right now.	现在
66	But most people don't don't even realise this is happening.	但大多数人甚至不知道这正在发生。
67	So some of the counter arguments might be, well, these are very, very smart people.	所以,有些辩驳可能,嗯, 这些是非常,非常聪明的人。
68	These are very big companies with lots of money.	这些公司是大公司 有很多钱。
69	They have a obligation and a moral obligation, but also just a legal obligation to make sure	他们有义务和道德义务,但也只是确保
70	they do no harm.	他们没有伤害。
71	So I'm sure it'll be fine.	所以,我敢肯定,这将是罚款。
72	The only obligation they have is to make money for the investors.	他们唯一的义务是为投资者赚钱。
73	That's the legal obligation they have.	这是他们的法律义务。
74	They have no moral or ethical obligations.	他们没有道德或道德义务。
75	Also, according to them, they don't know how to do it yet.	另外,根据他们的说法,他们还不知道该怎么做.
76	The state of the art answers are, we'll figure it out when we get there, or AI will help	艺术状态的答案是,我们到达那里后会想出办法,否则AI会帮忙
77	us control more advanced AI.	我们控制更先进的AI。
78	That's insane.	这是疯了。
79	In terms of probability, what do you think is the probability that something goes	在概率方面,你认为什么是 某些东西去的概率
80	catastrophically wrong?	灾难性的错误?
81	So nobody can tell you for sure what's going to happen.	所以没有人可以告诉你 确定会发生什么。
82	But if you're not in charge, you're not controlling it, you will not get outcomes you want.	但是,如果你不负责,你没有控制它,你不会得到你想要的结果.
83	The space of possibilities is almost infinite.	可能性的空间几乎无限。
84	The space of outcomes we will like is tiny.	我们所希望的成果空间很小。
85	And who are you and how long have you been working on this?	你是谁,你在这工作多久了?
86	I'm a computer scientist by training.	我是一个计算机科学家 通过训练。
87	I have a PhD in computer science and engineering.	我有一个计算机科学与工程博士。
88	I probably started work in AI safety, mildly defined as control of bots at the time, 15	我大概开始在AI安全工作, 轻度定义为控制bots 当时,15
89	years ago.	数年前.
90	15年前 15 years ago. 15 years ago. 15 years. 15 years ago. 15 years. 15 years.
91	So you've been working on AI safety before it was cool?	所以,你一直在工作 AI安全之前是很酷?
92	Before the term existed, I coined the term AI safety.	在这个词存在之前,我发明了AI安全一词.
93	So you're the founder of the term AI safety?	你是AI安全这个词的创始人?
94	The term, yes.	术语,是的。
95	Not the field.	不是田园.
96	There are other people who did brilliant work before I got there.	还有其他人在我到那里之前做了出色的工作。
97	Why were you thinking about this 15 years ago?	15年前你为什么考虑这个?
98	Because most people have only been talking about the term AI safety for the last two	因为大部分人只谈论 AI安全术语的最后两个
99	or three years.	或三年时间。
100	It started very mildly just as a security project.	一开始很温和 只是一个安全项目。
101	I was looking at poker bots.	我当时在看扑克棒
102	And I realized that the bots are getting better and better.	我意识到这些家伙越来越好。
103	And if you just project this forward enough, they're going to get better than us, smarter,	如果你能把事情做好 他们就会比我们聪明多了
104	more capable.	更厉害
105	And it happened.	事情就这样发生了
106	They are playing poker way better than average players.	他们玩扑克比普通球员好
107	But more generally, it will happen with all other domains, all the other cyber resources.	但更普遍的是,它会发生在所有其他领域, 所有其他的网络资源。
108	I wanted to make sure AI is a technology which is beneficial for everyone.	我想确保人工智能是一种有益于每个人的技术。
109	So I started work on making AI safer.	于是我开始努力让AI更安全.
110	Was there a particular moment in your career where you thought, oh, my God?	你职业生涯中是否有一个特殊的时刻 你想,哦,我的上帝?
111	First five years at least, I was working on solving this problem.	至少头五年,我在努力解决这个问题。
112	I was convinced we can make this happen.	我确信我们能做到这一点。
113	We can make safe AI.	我们可以做安全的人工智能。
114	And that was the goal.	这就是目标。
115	But the more I looked at it, the more I realized every single component of that equation	但我越看越了解 方程式的每个部分
116	is not something we can actually do.	并不是我们能做的
117	And the more you zoom in, it's like a fractal.	越放大,它就像一个分形。
118	You go in and you find 10 more problems and then 100 more problems.	你进去再找10个问题 然后再找100个问题
119	And all of them are not just difficult.	而且他们都不只是困难。
120	They're impossible to solve.	他们不可能解决的
121	There is no seminal work in this field where we solve this.	在这一领域没有开创性的工作,我们解决了这个问题。
122	We don't have to worry about this.	我们不必担心这一点。
123	There are patches.	有补丁.
124	There are little fixes we put in place and quickly people find ways to work around them.	我们没有多少办法 很快人们就会找到办法围绕它们工作。
125	They jailbreak whatever safety mechanisms we have.	他们越狱 任何我们的安全机制。
126	So while progress in AI capabilities is exponential or maybe even hyper-exponential,	因此,虽然AI能力的进步是指数性的,甚至可能是超责任性的,
127	progress in AI safety is linear or constant.	AI安全方面的进展是线性或恒定的。
128	The gap is increasing.	差距正在扩大。
129	The gap between how capable the systems are and how well we can control them,	这些系统的能力与我们控制它们的能力之间的差距,
130	predict what they're going to do, explain their decision making.	预测他们要做什么 解释他们的决策
131	I think this is quite an important point because you said that we're	我认为这很重要 因为你说过我们
132	basically patching over the issues that we find.	基本上解决我们发现的问题。
133	So we're developing this core intelligence and then to stop it doing things	因此,我们正在开发这个核心智能 然后阻止它做事情
134	or to stop it showing some of its unpredictability or its threats,	或阻止它表现出一些不可预测性或威胁,
135	the companies that are developing this AI are programming in code over the top	开发此AI的公司 正在编程代码 顶端
136	to say, okay, don't swear.	说,好吧,不要发誓。
137	Don't say that read word.	别这么说
138	Don't do that bad thing.	别做坏事
139	Exactly.	没错
140	And you can look at other examples of that.	你可以看看其他的例子。
141	So HR manuals, right?	所以人力资源手册,对不对?
142	We have those humans, their general intelligences,	我们有那些人类, 他们的一般智能,
143	but you want them to behave in a company.	但你想让他们在公司里表现
144	So they have a policy, no sexual harassment, no this, no that.	所以他们有政策,没有性骚扰,没有这个,没有那个。
145	But if you're smart enough, you always find a workaround.	但如果你够聪明 你总能找到工作
146	So you're just pushing behavior into a different, not yet restricted subdomain.	所以,你只是把行为推向 不同的,还没有限制的子域。
147	We should probably define some terms here.	我们也许应该在这里定义一些术语。
148	So there's narrow intelligence, which can play chess or whatever.	所以有狭义的智力,可以下棋什么的.
149	There's artificial general intelligence, which can operate across domains.	有人工一般智能,可以跨域操作.
150	And then super intelligence, which is smarter than all humans in all domains.	然后是超级智能,它比所有领域的人类都聪明。
151	And where are we?	我们在哪里?
152	So that's a very fuzzy boundary, right?	所以,这是一个非常模糊的边界,对不对?
153	We definitely have many excellent narrow systems, no question about it.	毫无疑问,我们有许多出色的狭小系统。
154	And they are super intelligent in that narrow domain.	它们在那个狭小的领域是超聪明的。
155	So protein folding is a problem which was solved using narrow AI,	所以蛋白质折叠是一个用狭义的AI来解决的问题,
156	and it's superior to all humans in that domain.	它优于这个领域的所有人类
157	In terms of AGI, again, I said, if we showed what we have today to a scientist from 20 years ago,	关于AGI,我再说一遍,如果我们能向一位20年前的科学家展示我们今天所拥有的东西,
158	they would be convinced we have full-blown AGI.	他们会相信我们有 完整的AGI。
159	We have systems which can learn, they can perform in hundreds of domains,	我们有能够学习的系统 它们可以在数百个领域表演
160	and they're better than human and many of them.	他们比人类和很多人都好
161	So you can argue we have a weak version of AGI.	所以,你可以论证我们有一个弱版本的AGI。
162	Now, we don't have super intelligence yet.	现在,我们还没有超级智能。
163	We still have brilliant humans who are completely dominating AI,	我们还有聪明的人类 完全主宰AI,
164	especially in science and engineering.	特别是在科学和工程方面。
165	But that gap is closing so fast.	但这一差距正在迅速缩小。
166	You can see, especially in the domain of mathematics.	你可以看到,特别是在数学领域。
167	Three years ago, large language models couldn't do basic algebra.	三年前,大语言模型无法做基本的代数.
168	Multiplying three-digit numbers was a challenge.	乘以三位数是一个挑战。
169	Now they're helping with mathematical proofs.	现在他们在帮助数学证明。
170	They're winning mathematics olympiads, competitions.	他们赢得了数学奥林匹亚奖,竞赛.
171	They're working on solving millennial problems, hardest problems in mathematics.	他们正在努力解决千年问题,数学中最难解决的问题.
172	So in three years, we closed the gap from subhuman performance	三年后,我们从非人类的表演中 缩小了差距
173	to better than most mathematicians in the world.	比世界上大多数数学家都要好
174	And we see the same process happening in science and in engineering.	我们在科学和工程方面也看到同样的过程。
175	You have made a series of predictions and they correspond to a variety of different dates,	你做了一系列的预测, 并且它们对应不同的日期,
176	and I have those dates in front of me here.	我前面还有那些日期
177	What is your prediction for the year 2027?	你对2027年的预测是什么?
178	We're probably looking at AGI as predicted by prediction markets and tops of the labs.	我们可能正在研究AGI, 预测市场和实验室的顶部。
179	So we'd have artificial general intelligence by 2027.	到2027年我们就会有人工一般情报
180	And how would that make the world different to how it is now?	那会如何让世界与现在不同?
181	So if you have this concept of a drop-in employee,	所以,如果你有这种概念 一个滴入员工,
182	you have free labor, physical and cognitive, trillions of dollars of it.	你有自由的劳动, 身体和认知,数万亿美元。
183	It makes no sense to hire humans for most jobs.	雇用人类从事大多数工作是毫无意义的。
184	If I can just get a $20 subscription or a free model to do what an employee does,	如果我能得到一个20美元订阅费或免费的模型做一个员工的工作,
185	first anything on a computer will be automated.	首先计算机上的任何东西都会自动化
186	And next, I think humanoid robots are maybe five years behind.	接着,我认为人造机器人可能落后五年。
187	So in five years, all the physical labor can also be automated.	所以在五年内,所有的体力劳动也可以自动化.
188	So we're looking at a world where we have levels of unemployment we've never seen before.	因此,我们所看到的是一个世界, 失业率是前所未有的。
189	Not talking about 10% unemployment, which is scary, but 99%.	不谈10%的失业,这是可怕的,但99%。
190	All you have left is jobs where, for whatever reason,	你所剩无几的工作 无论出于什么原因
191	you prefer another human would do it for you.	你更喜欢别人为你做
192	But anything else can be fully automated.	但其他的都可以完全自动化.
193	It doesn't mean it will be automated in practice.	这并不意味着它会在实际操作中自动化.
194	A lot of times, technology exists, but it's not deployed.	很多时候,技术是存在的,但没有部署.
195	Video phones were invented in the 70s.	视频手机在70年代发明.
196	Nobody had them until iPhones came around.	在iPhone出现之前没人有
197	So we may have a lot more time with jobs and with the world which looks like this.	因此,我们可能有更多的时间 与工作 和这样的世界。
198	But capability to replace most humans and most occupations will come very quickly.	但替代大多数人类和大多数职业的能力会很快出现.
199	Okay, so let's try and drill down into that and stress test it.	好吧,让我们尝试钻进那里 并压力测试它。
200	So a podcaster like me, would you need a podcaster like me?	所以像我这样的脚踏车 你需要我这样的脚踏车吗?
201	So let's look at what you do.	让我们看看你在做什么。
202	You prepare, you ask questions, you ask follow-up questions,	你准备,你问问题,你问后续问题,
203	and you look good on camera.	在镜头上你看起来不错。
204	Thank you so much.	非常感谢
205	Let's see what we can do.	试问如何是道.
206	Large language model today can easily read everything I wrote	今天的大语言模型可以很容易地阅读我写的每句话
207	and have very solid understanding.	并且有非常坚实的理解。
208	But I assume you haven't read every single one of my books.	但我猜你没看过我的每一本书
209	That thing would do it.	那东西会成功的
210	It can train on every podcast you ever did,	它可以训练每一个播客 你曾经做过,
211	so it knows exactly your style, the types of questions you ask.	所以它知道你的风格, 你问的问题的类型。
212	It can also find correspondence between what worked really well,	也能找到真正有效的东西之间的对应
213	like this type of question really increased views.	像这种问题真的增加了观点。
214	This type of topic was very promising, so you can optimize,	这种话题很有前途,你可以优化
215	I think better than you can because you don't have a data set.	我比你想象得更好 因为你没有数据集
216	Of course, visual simulation is trivial at this point.	当然,视觉模拟在这个时候是微不足道的.
217	So you can make a video within seconds of me sat here.	你可以在我坐在这里的几秒钟内拍个视频。
218	So we can generate videos of you interviewing anyone on any topic very efficiently,	这样我们就可以制作视频 有关你采访任何人 任何话题非常高效,
219	and you just have to get likeness approval, whatever.	你只需要得到像批准,随便什么。
220	Are there many jobs that you think would remain in a world of AGI?	你认为在AGI的世界里会留下很多工作吗?
221	If you're saying AGI is potentially going to be here,	如果你说AGI有可能在这里
222	whether it's deployed or not, by 2027,	无论是否部署, 到2027年,
223	what kind and then, okay, so let's take out of this any physical labor jobs for a second.	什么样的,然后, 好吧, 所以让我们从这里 任何体力劳动工作 一秒钟。
224	Are there any jobs that you think a human would be able to do better in a world of AGI still?	你认为在AGI的世界里 人类还能做得更好吗?
225	So that's the question I often ask people in the world with AGI,	所以我经常跟AGI一起问这个问题
226	and I think almost immediately we'll get superintelligence as a side effect.	我想我们马上就会得到超级情报 作为副作用
227	So the question really is in a world of superintelligence,	因此,问题真的是在一个超级智能的世界,
228	which is defined as better than all humans in all domains, what can you contribute?	它被定义为优于所有领域所有人类, 你能贡献什么?
229	And so you know better than anyone what it's like to be you.	让你比任何人都更清楚你是什么感觉
230	You know what ice cream tastes to you.	你知道什么是冰激凌味给你。
231	Can you get paid for that knowledge?	你能为这种知识得到报酬吗?
232	Is someone interested in that?	有人感兴趣吗?
233	Maybe not, not a big market.	也许不是,不是一个大市场。
234	There are jobs where you want a human.	有些工作你想要人类
235	Maybe you're rich and you want a human accountant for whatever historic reasons.	或许你很有钱 想要一个人类会计 不论历史原因
236	Old people like traditional ways of doing things.	老人喜欢传统的做事方式.
237	Warren Buffett would not switch to AI.	沃伦·巴菲特不会转投AI.
238	He would use his human accountant.	他会用他的人类会计。
239	But it's a tiny subset of a market.	但它是一个市场的小子集.
240	Today we have products which are man-made in the US,	今天我们有美国人造的产品
241	as opposed to mass-produced in China, and some people pay more to have those.	而不是中国的大规模生产, 有些人付出更多的钱来拥有这些。
242	But it's a small subset.	但它是一个小子集.
243	It's almost a fetish.	它几乎是一个迷。
244	There is no practical reason for it.	这样做没有任何实际理由。
245	And I think anything you can do on a computer could be automated using that technology.	我认为你对计算机所能做的任何事情 都可以使用这种技术实现自动化。
246	You must hear a lot of rebuttals to this when you say it,	你必须听到很多的反驳 当你说,
247	because people experience a huge amount of mental discomfort when they hear	因为人们在听到这些话时会感到精神不适
248	that their job, their career, the thing they got a degree in,	他们的工作,他们的事业, 他们得到了学位的东西,
249	the thing they invested $100,000 into is going to be taken away from them.	他们投资10万元的东西 将被夺走
250	So their natural reaction for some people is that cognitive dissonance,	他们的自然反应是认知不协调
251	that no, you're wrong.	不,你错了。
252	AI can't be creative.	AI不可能有创意.
253	It's not that.	不是那样的。
254	It'll never be interested in my job.	它永远不会对我的工作感兴趣。
255	I'll be fine because you hear these arguments all the time, right?	我会没事的,因为你总是听到这些争论,对不对?
256	It's really funny.	这真的很有趣。
257	I ask people and I ask people in different occupations.	我问人们,我问人们在不同的职业。
258	I ask my Uber driver, are you worried about self-driving cars?	我问我的Uber司机 你担心自驾车吗?
259	And they go, no.	他们去,没有。
260	No one can do what I do.	没有人能做我的工作。
261	I know the streets of New York.	我知道纽约的街道
262	I can navigate like no AI.	我可以像没有AI一样导航.
263	и
264	And it's true for any job.	亦复如是一切业.
265	Professors are saying this to me.	教授们在对我说这些
266	Nobody can lecture like I do.	没有人能像我一样讲课
267	This is so special.	这太特别了
268	But you understand it's ridiculous.	但你明白这很荒谬
269	We already have self-driving cars replacing drivers.	我们已经有自动驾驶的汽车 取代司机。
270	That is not even a question if it's possible.	如果有可能,那甚至不是一个问题。
271	It's like how soon before you're fired.	犹如早被炒.
272	Yeah, I mean, I've just been in LA yesterday and my car drives itself.	是啊,我的意思是,我昨天刚到洛杉矶 我的车自己开。
273	So I get in the car, I put in where I want to go,	所以我上车了,我放进我想去的地方
274	and then I don't touch the steering wheel or the brake pedals.	然后我不碰方向盘 或刹车踏板。
275	And it takes me from A to B,	这需要我从A到B
276	even if it's an hour-long drive without any intervention at all.	即使是一小时的车程 完全没有干预
277	I actually still park it.	其实我还是把车停了
278	But other than that, I'm not driving the car at all.	但除此之外,我根本没开车
279	I mean, obviously in LA, we also have Waymo now,	我的意思是,显然在洛杉矶, 我们现在也有Waymo,
280	which means you order it on your phone	也就是说你用手机订购
281	and it shows up with no driver in it and takes you to where you want to go.	它出现了,里面没有司机 并带你去你想去的地方。
282	So it's quite clear to see how that is potentially a matter of time.	因此,可以清楚地看到,这有可能是一个时间问题。
283	For those people,	对于这些人,
284	because we do have some of those people listening to this conversation right now,	因为我们确实有一些人 听这个谈话,现在,
285	that their occupation is driving.	他们的职业正在推动。
286	I think driving is the biggest occupation in the world, if I'm correct.	我认为驾驶是世界上最大的职业 如果我是对的
287	I'm pretty sure it is the biggest occupation in the world.	我敢肯定这是世界上最大的职业
288	What would you say to those people?	你会对那些人说什么?
289	What should they be doing with their lives?	他们的人生应该怎么办?
290	Should they be retraining in something or what time frame?	他们应该在什么时间范围内再培训?
291	So that's the paradigm shift here.	这就是这里的范式转变。
292	Before, we always said this job is going to be automated,	以前我们一直说这工作要自动化
293	retrained to do this other job.	重新训练完成其他任务
294	But if I'm telling you that all jobs will be automated,	但如果我告诉你所有的工作都是自动化的
295	then there is no Plan B.	那就没有B计划了
296	You cannot retrain.	你不能重新训练。
297	Look at computer science.	看看计算机科学
298	Two years ago, we told people, learn to code.	两年前,我们告诉人们,学会密码。
299	You are an artist.	你是个艺术家。
300	You cannot make money.	你不能赚钱。
301	Learn to code.	学会密码
302	Then we realized, oh, AI kind of knows how to code and getting better.	然后我们意识到,哦,AI 那种知道如何编码 和变得更好。
303	Become a prompt engineer.	成为一名迅速的工程师。
304	You can engineer prompts for AIs.	你可以为人工智能设计提示
305	It's going to be a great job.	将作大功德.
306	Get a four-year degree in it.	获得四年学位.
307	But then we're like, AI is way better at designing prompts for other AIs than any human.	但我们喜欢,AI比任何人类更擅长为其他AI设计提示.
308	So that's gone.	这么说就不见了
309	So I can't really tell you right now the hardest thing is design AI agents for practical applications.	所以我现在不能告诉你 最困难的就是 设计实用的人工智能
310	I guarantee you, in a year or two, it's going to be gone just as well.	我向你保证 一两年后 它也会消失的
311	So I don't think there is a...	所以我不认为有...
312	This occupation needs to learn to do this instead.	这种职业需要学会这样做。
313	I think it's more like, we as a humanity, then we all lose our jobs.	我觉得我们更像,作为一个人类, 然后我们都失去了我们的工作。
314	What do we do?	我们该怎么办?
315	What do we do financially?	我们财政上怎么办?
316	Who's paying for us?	谁为我们付钱?
317	And what do we do in terms of meaning?	我们该怎么办?
318	What do I do with my extra 60, 80 hours a week?	我每周多花60、80个小时怎么办?
319	You've thought around this corner, haven't you?	你想过这个角落吧?
320	A little bit.	一点点
321	What is around that corner, in your view?	在你看来,那个角落是什么?
322	So the economic part seems easy.	所以经济部分看起来很简单。
323	If you create a lot of free labor, you have a lot of free wealth,	如果你创造了很多自由劳动, 你有很多自由财富,
324	abundance, things which are right now not very affordable become very cheap.	丰产,现在买不起的东西变得非常便宜。
325	And so you can provide for everyone basic needs.	你可以满足每个人的基本需要。
326	Some people say you can provide beyond basic needs.	有些人说,你可以提供超出基本需求。
327	You can provide very good existence for everyone.	你可以为每个人提供非常好的生存。
328	The hard problem is, what do you do with all that free time?	困难的问题是,你做什么 这么多空闲时间?
329	For a lot of people, their jobs are what gives them meaning in their life.	对于很多人来说,他们的工作是他们生活中的意义所在.
330	So they would be kind of lost.	这样他们就会迷路了
331	We see it with people who retire or do early retirement.	我们从那些退休或提前退休的人那里看到这一点。
332	And for so many people who hate their jobs, they'll be very happy not working.	对于这么多讨厌工作的人来说,他们会很高兴不工作.
333	But now you have people who are chilling all day.	但是现在你的人 整天都在寒冷。
334	What happens to society?	社会会怎么样?
335	How does that impact crime rate, pregnancy rate, all sorts of issues?	这对犯罪率、怀孕率、各种问题有何影响?
336	Nobody thinks about it.	没有人会考虑的
337	Governments don't have programs prepared to deal with 99% unemployment.	政府没有制定解决99%失业问题的方案。
338	What do you think that world looks like?	你觉得这个世界是什么样子的?
339	Again, I think a very important part to understand here	再说一遍,我觉得这里有很重要的部分要理解
340	is the unpredictability of it.	是它的不可预测性。
341	We cannot predict what a smarter than us system will do.	我们不能预测一个比我们系统更聪明的系统会做什么。
342	And the point when we get to that is often called singularity	而当我们到达这一点 通常被称为奇点
343	by analogy with physical singularity.	与物理奇异性类比
344	You cannot see beyond the event horizon.	无法超越事件视野看到
345	I can tell you what I think might happen, but that's my prediction.	我可以告诉你我想会发生什么, 但我的预测。
346	It is not what actually is going to happen	事实并非如此
347	because I just don't have cognitive ability to predict	因为我只是没有 认知能力预测
348	a much smarter agent impacting this world.	一个更聪明的特工影响着这个世界
349	Then you read science fiction.	然后读科幻小说.
350	There is never a super intelligence in it actually doing anything	根本就没有什么超能力
351	because nobody can write believable science fiction at that level.	因为没有人能写出 令人难以置信的科幻小说
352	They either banned AI like Dune,	他们要么像沙丘一样禁止AI
353	because this way you can avoid writing about it.	因为这样你就可以避免写作
354	Or it's like Star Wars.	或如星战.
355	You have this really dumb bots, but nothing super intelligent ever.	你有一个非常愚蠢的家伙, 但从来没有超级聪明。
356	Because by definition, you cannot predict at that level.	因为从定义上讲,你不能预测到这个程度。
357	Because by definition of it being super intelligent,	因为它被定义为超智能,
358	it will make its own mind up.	它会决定自己的想法。
359	By definition, if it was something you could predict,	顾名思义,如果这是你能预测的
360	you would be operating at the same level of intelligence,	你将会以同样的情报水平行动
361	violating our assumption that it is smarter than you.	违反我们的假设 它比你聪明。
362	If I'm playing chess with super intelligence	如果我和超级智能一起下棋
363	and I can predict every move, I'm playing at that level.	我可以预测每一个动作, 我玩在那个水平。
364	It's kind of like my French bulldog trying to predict	这有点像我的法国斗牛犬试图预测
365	exactly what I'm thinking and what I'm going to do.	这正是我的想法和我要做的事。
366	That's a good cognitive gap.	这是一个很好的认知差距。
367	And it's not just he can predict you going to work,	不只是他能预测你会工作
368	you coming back, but he cannot understand why you're doing a podcast.	但是他不明白你为什么要做播客
369	That is something completely outside of his model of the world.	这完全超出了他的世界模式。
370	He doesn't even know that I go to work.	他连我去上班都不知道
371	He just sees that I leave the house and doesn't know where I go.	他只知道我离开家,不知道我去哪里
372	Buy food for him.	为他买饭.
373	What's the most persuasive argument against your own perspective here?	什么是最有说服力的论点 与你自己的观点在这里?
374	That we will not have unemployment due to advanced technology.	我们不会因为先进技术而失业。
375	That there won't be this French bulldog human gap in understanding	法国的斗牛犬人类不会在理解上出现空白
376	and I guess like power and control.	我猜是权力和控制权
377	So some people think that we can enhance human minds	所以有些人认为我们可以增强人的思想
378	either through combination with hardware,	要么通过结合硬件,
379	so something like Neuralink or through genetic re-engineering	所以像Neuralink 或者通过基因再造
380	to where we make smarter humans.	更聪明的人类
381	It may give us a little more intelligence.	可能会给我们一点智慧
382	I don't think we're still competitive in biological form with silicon form.	我认为我们还没有在生物形态上与硅形态竞争.
383	Silicon substrate is much more capable for intelligence.	硅基质对智能能力要强得多.
384	It's faster.	较快.
385	It's more resilient, more energy efficient in many ways.	它具有更强的韧性,在许多方面更有效率.
386	Which is what computers are made out of versus the brain.	电脑是用大脑来制造的
387	So I don't think we can keep up just with improving our biology.	所以,我认为我们不能跟上 仅仅改善我们的生物学。
388	Some people think maybe, and this is very speculative,	有些人认为,也许, 这是非常投机的,
389	we can upload our minds into computers.	我们可以把大脑上传到电脑上
390	So scan your brain, kind of comb of your brain	所以扫描你的大脑, 那种梳理你的大脑
391	and have a simulation running on a computer.	在电脑上进行模拟
392	And you can speed it up, give it more capabilities.	你可以加快速度,赋予它更多的能力。
393	But to me, that feels like you no longer exist.	但对我来说,感觉你已经不存在了
394	We just created software by different means	我们刚刚用不同的方法创造了软件
395	and now you have AI based on biology	现在你有了基于生物学的AI
396	and AI based on some other forms of training.	和大赦国际基于某些其他形式的培训。
397	You can have evolutionary algorithms.	你可以有进化算法。
398	You can have many paths to reach AGI.	您可以有许多途径可以到达 AGI 。
399	But at the end, none of them are humans.	但最后,他们都不是人类.
400	I have another date here, which is 2030.	我这里还有一个约会,是2030年。
401	What's your prediction for 2030?	你对2030年有什么预测?
402	What will the world look like?	世界会变成什么样?
403	So we probably will have humanoid robots	所以我们可能会有人造机器人
404	with enough flexibility, dexterity to compete with humans	足够灵活 灵活与人类竞争
405	in all domains, including plumbers.	在所有领域,包括水管工。
406	We can make artificial plumbers.	我们可以做人工水管工。
407	Not the plumbers.	不是水管工,不是他们
408	That felt like the last bastion of human employment.	这感觉就像人类就业的最后堡垒。
409	So 20, 30, five years from now, humanoid robots.	因此,20,30,5年后,人类机器人。
410	So many of the companies, the leading companies,	这么多的公司, 领先的公司,
411	including Tesla, are developing humanoid robots at light speed	包括特斯拉,正在以光速开发人造机器人
412	and they're getting increasingly more effective.	他们越来越有效。
413	These humanoid robots will be able to move through physical space.	这些人造机器人将能够通过物理空间移动.
414	Or, you know, make an omelette, do anything humans can do,	或者,你知道,做一个煎饼, 做任何人类可以做的,
415	but obviously have, be connected to AI as well.	但显然也和AI有关
416	So they can think, talk.	所以他们可以思考,说话。
417	Right, they're controlled by AI.	没错 他们被AI控制着
418	They're always connected to the network.	他们总是与网络相连.
419	So they're already dominating in many ways.	所以他们已经在很多方面占据主导地位。
420	Our world will look remarkably different	我们的世界会变得很不一样
421	when humanoid robots are functional and effective.	当人造机器人的功能性和有效性。
422	Because that's really when, you know, I start to cry.	因为那时我开始哭
423	The combination of intelligence and physical ability	智力和身体能力的结合
424	is really, really doesn't leave much, does it, for us human beings?	真的,真的没有留下很多, 是不是,我们人类?
425	Not much.	没什么
426	So today, if you have intelligence through internet,	所以今天 如果你通过互联网获得情报
427	you can hire humans to do your bidding for you.	你可以雇人来为你做你的工作
428	You can pay them in Bitcoin.	你可以用比特币支付
429	So you can have bodies just not directly controlling them.	所以,你可以有身体 只是不直接控制它们。
430	So it's not a huge game changer to add direct control of physical bodies.	因此,加入对物理体的直接控制并不是一个巨大的游戏改变器.
431	Intelligence is where it's at.	智是处.
432	The important component is definitely higher ability to optimize,	关键是提高优化的能力
433	to solve problems, to find patterns people cannot see.	寻找人们看不到的规律
434	And then by 2045, I guess the world looks even more,	到了2045年,我猜世界看起来更像
435	which is 20 years from now.	这是20年后。
436	So if it's still around?	如果还在附近呢?
437	If it's still around.	若还围绕.
438	Ray Kurzweil predicts that that's the year for the singularity.	雷·库尔兹韦尔(Ray Kurzweil)预测,今年是奇数的一年.
439	That's the year where progress becomes so fast.	那是进展如此迅速的一年。
440	So this AI doing science and engineering work	所以这个AI做科学和工程工作
441	makes improvements so quickly we cannot keep up anymore.	如此快的改善,我们不能再跟上。
442	That's the definition of singularity,	这就是奇异性的定义,
443	point beyond which we cannot see, understand, predict.	这一点我们无法再看到、理解、预测。
444	See and understand, predict the intelligence itself or?	看和理解,预测情报本身还是?
445	What is happening in the world?	到底发生了什么?
446	The technology is being developed.	该技术正在开发中。
447	So right now, if I have an iPhone,	所以,现在,如果我有一个iPhone,
448	I can look forward to a new one coming out next year,	我可以期待明年有一个新的
449	and I'll understand it has slightly better camera.	我将明白它有 稍好一些的相机。
450	Imagine now this process of researching and developing	想象一下现在这个研究与发展的过程
451	this phone is automated.	电话是自动的
452	It happens every six months, every three months,	每六个月,每三个月
453	every month, week, day, hour, minute, second.	每个月,一周,一天,小时,分钟,第二.
454	You cannot keep up with 30 iterations of iPhone in one day.	您无法在一天之内保持 30 个 iPhone 的迭代 。
455	You don't understand what capabilities it has,	你不知道它有什么样的能力,
456	what proper controls are.	什么是适当的控制。
457	It just escapes you.	它只是逃脱了你。
458	Right now, it's hard for any researcher in AI	现在对AI的研究人员来说很难
459	to keep up with the state of the art.	以跟上最新艺术。
460	While I was doing this interview with you,	在我接受你采访时
461	a new model came out,	一个新的模型出来了,
462	and I no longer know what the state of the art is.	我不再知道什么是 艺术状态。
463	Every day, as a percentage of total knowledge, I get dumber.	每天,作为总知识的百分率,我变得更笨。
464	I may still know more because I keep reading,	我可能还知道更多 因为我一直在看
465	but as a percentage of overall knowledge,	但作为总知识的百分比,
466	we're all getting dumber.	我们都越来越蠢了。
467	And then you take it to extreme values,	然后你把它到极端的价值,
468	you have zero knowledge,	你们毫无知识,
469	zero understanding of the world around you.	对你周围的世界一无所知
470	Some of the arguments against this eventuality	反对这种可能性的一些论点
471	are that when you look at other technologies	当你看其他技术时
472	like the Industrial Revolution,	像工业革命,
473	people just found new ways to work	人们刚刚找到新的工作方式
474	and new careers that we could never have imagined	我们无法想象的新事业
475	at the time were created.	当时,
476	How do you respond to that in a world of superintelligence?	在一个超级情报的世界里,你对此有什么反应?
477	It's a paradigm shift.	范式转变.
478	We always had tools, new tools,	我们总是有工具 新的工具
479	which allowed some job to be done more efficiently,	这使得一些工作能够更有效地完成,
480	so instead of having 10 workers,	而不是有10个工人,
481	you could have two workers,	你可以有两个工人,
482	and eight workers had to find a new job.	8名工人不得不找到新的工作。
483	And there was another job,	还有一份工作
484	now you can supervise those workers	现在你可以监督那些工人
485	or do something cool.	或者做点很酷的事
486	If you're creating a meta invention,	如果你正在创造一个元发明,
487	you're inventing intelligence,	你在创造情报
488	you're inventing a worker, an agent,	你在发明一个工人,一个经纪人,
489	then you can apply that agent to the new job.	然后,你可以应用那个代理 到新的工作。
490	There is not a job which cannot be automated.	没有一项工作不能自动化。
491	That never happened before.	这以前从来没有发生过。
492	All the inventions we previously had	我们以前所有的发明
493	were kind of a tool for doing something.	是一种做一些事情的工具。
494	So we invented fire, huge game changer,	所以我们发明了火,巨大的游戏改变者,
495	but that's it, it stops with fire.	但就是这样,它用火停止。
496	We invent a wheel, same idea, huge implications,	我们发明一个轮子,同样的想法, 巨大的影响,
497	but wheel itself is not an inventor.	但轮子本身不是发明家
498	Here we're inventing a replacement for human mind,	在这里,我们正在发明一个替代 人的心灵,
499	a new inventor capable of doing new inventions.	一个能做新发明的新发明家
500	It's the last invention we ever have to make.	这是我们最后的发明了
501	At that point, it takes over.	这时,它接管。
502	And the process of doing science, research,	和做科学,研究的过程,
503	even ethics research, morals,	甚至道德研究、道德
504	all that is automated at that point.	所有那些都是自动化的
505	Do you sleep well at night?	你晚上睡得好吗?
506	Really well.	不错
507	Even though you spent the last 15, 20 years of your life	虽然你花了你生命的最后15年,20年
508	working on AI safety,	在人工智能安全方面工作,
509	and it's suddenly among us in a way	它突然出现在我们中间
510	that I don't think anyone could have predicted five years ago.	我想五年前没人能预料到
511	I want to say among us,	我想说,在我们中间,
512	I really mean that the amount of funding and talent	我真的是指资金和人才的数量
513	that is now focused on reaching super intelligence faster	现在的重点是更快地达到超级智能
514	has made it feel more inevitable and more soon	使感觉更加难免 更快
515	than any of us could have possibly imagined.	比我们任何人都想象不到的
516	We as humans have this built-in bias	我们人类有这种内在偏见
517	about not thinking about really bad outcomes	没有想到真正的坏结果
518	and things we cannot prevent.	以及我们无法阻止的事
519	So all of us are dying.	故一切众生皆死.
520	Your kids are dying.	你的孩子们快死了
521	Your parents are dying.	汝父母临终.
522	Everyone's dying.	诸人皆死.
523	But you still sleep well.	但你睡得还好
524	You still go on with your day.	你还在继续你的一天。
525	Even 95-year-olds are still doing games	连95岁的孩子都还在玩游戏
526	and playing golf and whatnot.	并打高尔夫球什么的。
527	Because we have this ability to not think	因为我们有这种能力 不去思考
528	about the worst outcomes,	关于最糟糕的结果,
529	especially if we cannot actually modify the outcome.	特别是如果我们不能改变结果。
530	So that's the same infrastructure being used for this.	所以,这是同样的基础设施 正在用于这个。
531	Yeah, there is humanity level,	是的,有人类层面,
532	death-like event.	类似死亡的事件。
533	We're happening to be close to it probably.	我们正巧接近它 可能。
534	But unless I can do something about it,	但是除非我能做点什么
535	I can just keep enjoying my life.	我可以继续享受我的生活。
536	In fact, maybe knowing that you have limited amount of time left	其实,也许知道 你的时间还有限
537	gives you more reason to have a better life.	让你有更好的生活
538	You cannot waste any.	你不能浪费任何东西。
539	And that's the survival trait of evolution, I guess,	我猜这就是进化论的生存特征
540	because those of my ancestors	因为我的祖先
541	that spent all their time worrying	花了他们所有的时间 担心
542	wouldn't have spent enough time	不会花足够的时间
543	having babies and hunting to survive.	有了婴儿和狩猎生存。
544	Suicidal ideation,	自杀的想法,
545	people who really start thinking about	真正开始思考的人
546	how horrible the world is,	这个世界是多么可怕,
547	usually escape pretty soon.	通常会很快逃脱
548	You co-authored this paper,	你联合撰写了这份论文,
549	analyzing the key arguments	分析关键论据
550	people make against the importance of AI safety.	人们反对AI安全的重要性。
551	And one of the arguments in there is that	其中的一个论点是
552	there's other things that are of bigger importance right now.	现在还有更重要的事
553	It might be world wars.	也许是世界大战
554	It could be nuclear containment.	这可能是核遏制。
555	It could be other things.	也可能是别的
556	There's other things that the governments	还有其他事情 政府
557	and podcasters like me should be talking about	和我这样的播客应该谈论
558	that are more important.	这更重要
559	What's your rebuttal to that argument?	你对这个论点有什么反驳?
560	Superintelligence is a meta solution.	超级智能是一种元解.
561	If we get superintelligence right,	如果我们得到超级情报的权利,
562	it will help us with climate change.	这将帮助我们应对气候变化。
563	It will help us with wars.	这将帮助我们进行战争。
564	It can solve all the other existential risks.	它可以解决其他所有的存在风险.
565	If we don't get it right, it dominates.	如果我们不正确,它会占上风。
566	If climate change will take 100 years to boil us alive	如果气候变化要100年才能让我们活活烧死
567	and superintelligence kills everyone in five,	和超级情报 杀死每个人在五,
568	I don't have to worry about climate change.	我不必担心气候变化。
569	So either way, either it solves it for me	所以不管怎么样 要么就解决了
570	or it's not an issue.	或者这不是一个问题。
571	So you think it's the most important thing	所以你认为这是最重要的事情
572	to be working on?	工作吗?
573	Without question, there is nothing more important	没有比这更重要的了
574	than getting this right.	比得到这个正确。
575	And I know everyone says that you take any class,	我知道每个人都说你上任何课程,
576	but you take English professor's class	但你会上英语教授的课
577	and he tells you this is the most important class	他告诉你这是最重要的班级
578	you'll ever take.	你会永远接受。
579	But you can see the meta level differences with this one.	但是,你可以看到元级别 差异与这个。
580	Another argument in that paper is that	该文件的另一个论点是:
581	we'll be in control and that the danger is not AI.	危险不是AI
582	This particular argument asserts that AI is just a tool.	这一具体论点断言AI只是一个工具.
583	Humans are the real actors that present danger	人类才是真正的危险人物
584	and we can always maintain control by simply turning it off.	我们总是可以保持控制 通过关闭它。
585	Can't we just pull the plug out?	我们不能把插头拔掉吗?
586	I see that every time we have a conversation on the show	我每一次在节目里聊天
587	about AI, someone says, can't we just unplug it?	关于AI,有人说, 我们不能只是解开它?
588	Yeah, I get those comments on every podcast I make	对,每个播客都这么说
589	and I always want to like get in touch with a guy	我一直想和男人联系
590	and say, this is brilliant.	并说,这是辉煌的。
591	I never thought of it.	我从来没想过
592	We're going to write a paper together	我们要一起写一篇论文
593	and get a Nobel Prize for it.	并因此获得诺贝尔奖.
594	This is like, let's do it because it's so silly.	这就像,让我们做到这一点,因为它是如此愚蠢。
595	Can you turn off a virus?	你能关掉病毒吗?
596	You have a computer virus, you don't like it.	你有一个电脑病毒,你不喜欢它。
597	Turn it off.	关掉
598	How about Bitcoin?	比特币怎么样?
599	Turn off Bitcoin network.	关闭比特币网.
600	Go ahead, I'll wait.	来吧,我等着
601	This is silly.	这是愚蠢的。
602	Those are distributed systems.	这些是分布式系统。
603	You cannot turn them off and on top of it,	你不能把它们关掉 并且在上面,
604	they're smarter than you.	他们比你聪明
605	They made multiple backups.	他们做了多个备份。
606	They predicted what you're going to do.	他们预测你会怎么做
607	They will turn you off before you can turn them off.	他们会把你关起来,然后你才能关掉
608	The idea that we will be in control applies only to	我们控制的想法只适用于
609	pre-superintelligence levels, basically what we have today.	情报前的水平, 基本上我们今天有。
610	Today, humans with AI tools are dangerous.	今天,带有AI工具的人类是危险的.
611	They can be hackers, malevolent actors, absolutely.	他们可以是黑客,邪恶的演员,绝对。
612	But the moment superintelligence becomes smarter, dominates,	但当超级情报变得更聪明 主导
613	they're no longer the important part of that equation.	他们不再是这个方程式的重要部分了
614	It is the higher intelligence I'm concerned about,	这是我所担心的高级情报
615	not the human who may add additional malevolent payload,	而不是增加恶意载荷的人,
616	but at the end still doesn't control it.	但最后还是无法控制
617	It is tempting to follow the next argument	接下来的争论很诱人
618	that I saw in that paper, which basically says,	我在报纸上看到, 基本上说,
619	listen, this is inevitable.	听着,这是不可避免的。
620	So there's no point fighting against it because	所以,没有意义与之斗争,因为
621	there's really no hope here.	这里真的没有希望
622	So we should probably give up even trying	所以,我们也许应该放弃 甚至尝试
623	and be faithful that it will work itself out	并谨守信念,以便它成功,
624	because everything you've said sounds really inevitable.	因为你所说的一切听起来都是不可避免的
625	And with China working on it, I'm sure Putin's got some	有了中国,我相信普京有
626	secret division.	秘密组织
627	I'm sure Iran are doing some bits and pieces.	我敢肯定,伊朗正在做一些零碎。
628	Every European country's trying to get ahead of AI.	每个欧洲国家都试图超越AI.
629	The United States is leading the way.	美国正在领先。
630	So it's inevitable.	所以这是不可避免的。
631	So we probably should just have faith and pray.	所以我们应该有信仰和祈祷。
632	Praying is always good, but incentives matter.	祈祷总是好的,但激励很重要。
633	If you are looking at what drives these people.	如果你在看什么驱使着这些人
634	So yes, money is important.	是的,钱很重要。
635	So there is a lot of money in that space.	所以那里有很多钱
636	And so everyone's trying to be there and develop this technology.	所以,每个人都在努力去开发这个技术。
637	But if they truly understand the argument,	但是如果他们真的理解这个论点
638	they understand that you will be dead.	他们知道你会死的
639	No amount of money will be useful to you.	没有钱对你有用
640	Then incentive switch.	然后激励转换。
641	They would want to not be dead.	他们不想死
642	A lot of them are young people, rich people.	很多人都是年轻人,有钱人.
643	They have their whole lives ahead of them.	他们有他们的全部生活 在他们之前。
644	I think they would be better off not building advanced	我想他们最好不要建高楼
645	superintelligence, concentrating on narrow AI tools	超智能,专注于狭义的AI工具
646	for solving specific problems.	用于解决具体问题。
647	Okay, my company cures breast cancer.	好吧,我的公司治疗乳腺癌。
648	That's all.	就这样
649	We make billions of dollars.	我们赚了几十亿美元。
650	Everyone's happy.	皆悉欢喜.
651	Everyone benefits.	每个人都有益处。
652	It's a win.	成胜义.
653	We are still in control today.	我们今天仍在控制之中。
654	It's not over until it's over.	终不灭前.
655	We can decide not to build general superintelligences.	我们可以决定不建立一般的超级情报机构.
656	The United States might be able to conjure up enough enthusiasm for that.	美国也许能够为此激发足够的热情。
657	But if the United States doesn't build general superintelligences,	但如果美国不建立一般的超级情报,
658	then China are going to have the big advantage, right?	那么中国将拥有很大的优势,对吧?
659	So right now, at those levels, whoever has more advanced AI	因此,现在,在这些层次, 谁有更先进的AI
660	has more advanced military.	有更先进的军事。
661	No question.	没问题
662	We see it with existing conflicts.	我们看到的是现有的冲突。
663	But the moment you switch to superintelligence	但你一转身到超级情报
664	and control superintelligence, it doesn't matter who builds it,	控制超级情报,不管是谁造的
665	us or them.	我们或他们。
666	And if they understand this argument,	如果他们理解这个论点
667	they also would not build it.	他们也不会建造它。
668	It's a mutually assured distraction on both ends.	这在两端都是相互保证的分心
669	Is this technology different than, say, nuclear weapons,	这种技术是否不同于核武器?
670	which require a huge amount of investment	需要大量投资
671	and you have to like enrich the uranium	你必须喜欢浓缩铀
672	and you need billions of dollars potentially	你可能需要数十亿美元
673	to even build a nuclear weapon?	甚至制造核武器?
674	But it feels like this technology is much cheaper	但觉得这种技术便宜得多
675	to get to superintelligence potentially,	有可能进入超级情报系统
676	or at least it will become cheaper.	或至少会变得便宜。
677	I wonder if it's possible that some guy,	不知道是不是某个人
678	some startup is going to be able to build superintelligence	一定能造出超级情报
679	in a couple of years without the need of billions of dollars	几年内不需要数十亿美元
680	of compute or electricity power.	计算或电力。
681	That's a great point.	说得好
682	So every year it becomes cheaper and cheaper	所以,每年它变得便宜和便宜
683	to train sufficiently large model.	训练足够大的模型。
684	If today it would take a trillion dollars	如果今天需要一万亿美元
685	to build superintelligence, next year it could be a hundred billion	明年可能要一千亿
686	and so on.	诸如来.
687	At some point, a guy in a laptop could do it.	在某个时候,一个手提电脑的人可以做到这一点。
688	But you don't want to wait four years to make it affordable.	但你不想等4年 让它负担得起。
689	So that's why so much money is pouring in.	所以才有这么多的钱
690	Somebody wants to get there this year	今年有人想赶到那里
691	and lucky and all the winnings, light cone level award.	幸运和所有的胜利, 轻锥级奖。
692	So in that regard, they both very expensive projects	因此,在这方面,它们都非常昂贵的项目
693	like Manhattan level projects.	就像曼哈顿计划
694	Which was the nuclear bomb project.	那是核弹计划
695	Right.	对
696	The difference between the two technologies	两种技术的区别
697	is that nuclear weapons are still tools.	核武器仍然是工具。
698	Some dictators, some countries,	一些独裁者,一些国家,
699	someone has to decide to use them, deploy them.	总得有人决定使用 部署
700	Whereas superintelligence is not a tool, it's an agent.	虽然超级情报不是工具,但它是一个代理。
701	It makes its own decisions and no one is controlling it.	它自己作出决定,没有人控制它。
702	They cannot take out this dictator	他们不能干掉这个独裁者
703	and now superintelligence is safe.	现在超级情报安全了
704	So that's a fundamental difference to me.	这对我来说是一个根本的区别。
705	But if you're saying that it is going to get incrementally cheaper,	但如果你说会越来越便宜
706	like I think it's Moore's law, isn't it?	就像我认为这是摩尔的法律,不是吗?
707	That technology gets cheaper.	这种技术越来越便宜。
708	Then there is a future where some guy on his laptop	将来有人在笔记本电脑上
709	is going to be able to create superintelligence	就能创造出超级情报
710	without oversight or regulation or employees, et cetera.	没有监督或监管或雇员,等等。
711	Yeah.	对
712	That's why a lot of people suggesting	这就是为什么很多人建议
713	we need to build something like surveillance planet	我们需要建造一些像监视星球的东西
714	where you are monitoring who's doing what	监视谁在做什么
715	and you're trying to prevent people from doing it.	你试图阻止人们这样做。
716	Do I think it's feasible?	我觉得可行吗?
717	No.	没有
718	At some point it becomes so affordable and so trivial	在某个时候,它变得如此便宜,如此微不足道
719	that it just will happen.	它会发生。
720	But at this point, we're trying to get more time.	但此时,我们正努力争取更多的时间.
721	We don't want it to happen in five years.	我们不希望在五年内发生。
722	We want it to happen in 50 years.	我们希望在50年后发生。
723	I mean, that's not very hopeful.	我的意思是,这不是很希望。
724	Depends on how old you are.	这取决于你多大了。
725	I mean, if you're saying that you believe in the future	我是说 如果你是说你相信未来
726	people will be able to make superintelligence	人们可以制造超级情报
727	without the resources that are required today,	没有今天所需要的资源,
728	then it is just a matter of time.	那只是时间问题
729	Yeah.	对
730	But so will be true for many other technologies.	但许多其他技术也是如此。
731	We're getting much better in synthetic biology	我们的合成生物学越来越好
732	where today someone with a bachelor's degree in biology	今天有生物学学士学位的人
733	can probably create a new virus.	可能会产生新的病毒
734	This will also become cheaper.	这也会更便宜。
735	Other technologies like that.	其他技术 类似。
736	So we are approaching a point where it's very difficult	所以,我们正接近一个非常困难的点
737	to make sure no technological breakthrough is the last one.	确保没有技术突破是最后一个。
738	So essentially in many directions we have this pattern	因此,基本上在许多方向 我们有这种模式
739	of making it easier in terms of resources,	使资源更加容易,
740	in terms of intelligence to destroy the world.	以智灭世.
741	If you look at, I don't know, 500 years ago,	如果你看,我不知道, 500年前,
742	the worst dictator with all the resources	最糟糕的独裁者 拥有所有的资源
743	could kill a couple million people.	可能杀死几百万人。
744	He couldn't destroy the world.	他无法毁灭世界
745	Now we know nuclear weapons,	现在我们知道了核武器,
746	we can blow up the whole planet multiple times over.	我们可以炸遍整个星球
747	Synthetic biology, we saw with COVID,	合成生物学,我们看到COVID,
748	you can very easily create a combination virus	你很容易生成一种病毒
749	which impacts billions of people.	影响数十亿人。
750	And all of those things becoming easier to do.	而所有这些事情 变得更容易做。
751	In the near term, you talk about extinction being a real risk,	近期内,你说灭绝是一种真正的风险,
752	human extinction being a real risk.	人类灭绝是一个真正的风险。
753	Of all the pathways to human extinction	在人类灭绝的所有途径中
754	that you think are most likely, what is the leading pathway?	你认为最有可能的是什么主要路径?
755	Because I know you talk about there being	因为我知道你在说
756	some issue pre-deployment of these AI tools,	这些AI工具在部署前有些问题,
757	someone makes a mistake when they're designing a model	有人在设计模型时犯了错误
758	or other issues post-deployment.	或其他问题。
759	When I say post-deployment, I mean,	当我说部署后,我的意思是,
760	once a Chachapiti or something,	曾经是查查皮提之类的
761	an agent's released into the world	一个特工被释放到世界
762	and someone hacking into it and changing it	和有人黑进并改变它
763	and reprogramming it to be malicious.	并重新编程为恶意.
764	Of all these potential paths to human extinction,	在所有这些人类灭绝的可能途径中,
765	which one do you think is the highest probability?	你觉得哪一个是最大的概率?
766	So I can only talk about the ones I can predict myself.	所以我只能说那些我能预测的
767	So I can predict even before we get to super intelligence,	因此,我可以预测,甚至在我们到达超级智能之前,
768	someone will create a very advanced biological tool,	有人会创造出一种非常先进的生物工具,
769	create a novel virus,	创造新病毒
770	and that virus gets everyone or most everyone.	病毒会让所有人 或大多数人。
771	I can envision it, I can understand the pathway,	我能想象,我能理解路径,
772	I can say that.	我可以这么说
773	So just to zoom in on that then,	所以为了放大这个
774	that would be using an AI to make a virus	利用人工智能制造病毒
775	and then releasing it.	然后释放出来
776	And would that be intentional?	这是故意的吗?
777	There is a lot of psychopaths,	这里有很多精神病患者
778	a lot of terrorists, a lot of doomsday calls.	很多恐怖分子, 很多末日的电话。
779	We've seen historically, again,	我们又在历史上见过
780	they tried to kill as many people as they can,	他们试图杀死尽可能多的人,
781	they usually fail, they kill hundreds of thousands.	他们通常失败, 他们杀死数十万人。
782	But if they get technology to kill millions of billions,	但是,如果他们得到技术 杀死数百万亿,
783	they would do that gladly.	他们会很乐意的
784	The point I'm trying to emphasize	我想强调的一点
785	is that it doesn't matter what I can come up with.	无所谓我有什么办法
786	I am not a malevolent actor you're trying to defeat here.	我不是一个恶毒的演员 你试图打败这里。
787	It's the super intelligence which can come up	是超级智能能出现
788	with completely novel ways of doing it.	以全新的方法做到这一点。
789	Again, you brought up example of your dog.	你再次举出你的狗的例子
790	Your dog cannot understand all the ways	你的狗无法理解一切
791	you can take it out.	你可以把它拿出来。
792	It can maybe think you'll bite it to death or something.	它可能认为你会咬死它什么的。
793	But that's all.	仅此而已
794	Whereas you have infinite supply of resources.	而你却有无限的资源供给
795	So if I asked your dog exactly how you're going to take it out,	所以,如果我问你的狗 确切地说,你会如何把它拿出来,
796	it would not give you a meaningful answer.	它不会给你一个有意义的答案。
797	It can talk about biting.	它可以谈论咬人。
798	And this is what we know, we know viruses,	这就是我们知道,我们知道病毒,
799	we experienced viruses, we can talk about them.	我们经历了病毒,我们可以谈论它们。
800	But what an AI system capable of doing	但AI系统能做什么
801	novel physics research can come up with	新的物理学研究可以提出
802	is beyond me.	已经超越了我
803	One of the things that I think most people don't understand	我觉得大多数人都不明白的一件事
804	is how little we understand	我们了解得很少
805	about how these AIs are actually working.	关于这些人工智能的实际作用。
806	Because one would assume with computers,	因为用电脑可以假设
807	we kind of understand how a computer works.	我们有点理解 计算机是如何工作的。
808	We know that it's doing this and then this	我们知道它这样做 然后这个
809	and it's running on code.	并运行在代码。
810	But from reading your work,	但从读你的作品,
811	you described it as being a black box.	你形容它是一个黑匣子。
812	In the context of something like ChatGBT or an AI we know,	在类似ChatGBT或人工智能的情况下
813	you're telling me that the people that have built that tool	你是在告诉我那些造工具的人
814	don't actually know what's going on inside there.	不知道里面发生了什么
815	That's exactly right.	没错
816	So even people making those systems	所以即使是制造这些系统的人
817	have to run experiments on their product	必须在他们的产品上进行实验
818	to learn what it's capable of.	学习它的能力。
819	So they train it by giving it all of data,	他们通过提供所有的数据来训练它,
820	let's say all of internet text.	说到所有互联网短信
821	They run it on a lot of computers	他们在很多电脑上运行
822	to learn patterns in that text.	在文本中学习规律。
823	And then they start experimenting with that model.	然后他们开始试验这个模型。
824	Or do you speak French?	或者你会说法语?
825	Or can you do mathematics?	也可以做数学?
826	Or are you lying to me now?	还是你现在在骗我?
827	And so maybe it takes a year to train it	所以也许训练它需要一年时间
828	and then six months to get some fundamentals	然后6个月 得到一些基本
829	about what it's capable of, some safety overhead.	关于它的能力, 一些安全顶端。
830	But we still discover new capabilities in old models.	但我们仍然在旧模型中发现了新的能力.
831	If you ask the question in a different way,	如果你用不同的方式问问题
832	it becomes smarter.	它变得更聪明。
833	So it's no longer engineering	所以这已经不是工程了
834	how it was the first 50 years	前50年怎么样?
835	where someone was a knowledge engineer,	在那里有人是一个知识工程师,
836	programming an expert system AI to do specific things.	编程一个专家系统AI来做具体的事情.
837	It's a science.	科举.
838	We are creating this artifact, growing it.	我们正在创造这个文物, 种植它。
839	It's like an alien plant.	如异类植物.
840	And then we study it to see what it's doing.	然后我们研究它,看看它在做什么。
841	And just like with plants,	就像植物一样,
842	we don't have 100% accurate knowledge of biology.	我们没有100%的准确的生物学知识。
843	We don't have full knowledge here.	我们在这里没有完全的知识。
844	We kind of know some patterns.	我们知道一些规律。
845	We know, okay, if we add more compute,	我们知道,好吧,如果我们加更多的计算,
846	it gets smarter most of the time.	大部分时间都变聪明了
847	But nobody can tell you precisely	但没人能准确告诉你
848	what the outcome is going to be given a set of inputs.	将给结果一系列投入。
849	I've watched so many entrepreneurs	我看过这么多企业家
850	treat sales like a performance problem	把销售当成一个性能问题
851	when it's often down to visibility.	当它经常降低到能见度。
852	Because when you can't see	因为当你看不见
853	what's happening in your pipeline,	你的管道里发生了什么
854	what stage each conversation is at,	每次谈话都在哪个阶段?
855	what's stalled, what's moving,	什么是停滞, 什么是移动,
856	you can't improve anything	你什么都改进不了
857	and you can't close the deal.	你不能结束交易。
858	我们的赞助商PipeDrive 是第一CRM工具
859	for small to medium businesses.	中小企业。
860	Not just a contact list,	不只是联系人名单
861	but an actual system that shows your entire sales process	但是一个显示你整个销售过程的实际系统
862	end to end, everything that's live,	结束到结束, 一切活着,
863	what's lagging and the steps you need to take next.	落后的是什么 下一步你需要采取的步骤
864	All of your teams can move smarter and faster.	你们的队伍可以更聪明更快地前进.
865	Teams using PipeDrive are on average closing	使用管道Drive的队伍平均关闭
866	three times more deals than those that aren't.	3倍于那些不是交易。
867	It's the first CRM made by salespeople for salespeople	这是销售员为销售员制作的第一个CRM
868	that over 100,000 companies around the world rely on,	全世界有10万多家公司依赖,
869	including my team who absolutely love it.	包括我的团队谁绝对喜欢它。
870	通过访问pipeDrive.com slash CEO,今天给PipeDrive一个尝试.
871	And you can get up and running in a couple of minutes	你可以在几分钟内起来跑步
872	with no payment needed.	无需支付。
873	And if you use this link,	如果你用这个链接,
874	you'll get a 30-day free trial.	你会得到一个30天的自由审判。
875	你觉得OpenAI和Sam Altman怎么样?
876	and what they're doing?	他们在做什么?
877	And obviously you're aware that one of the co-founders,	显然你知道 其中一个联合创始人
878	was it Ilya Jack?	是伊利亚·杰克吗?
879	Ilya, yeah.	伊利亚,是的。
880	Ilya left and he started a new company called?	伊利亚走了 他开了一家新公司
881	Super Intelligent Safety.	超级智能安全.
882	Because AI safety wasn't challenging enough,	因为AI安全不够有挑战性
883	he decided to just jump right to the hard problem.	他决定直奔难题
884	As an onlooker, when you see that people are leaving OpenAI	当你看到人们离开OpenAI的时候
885	to start Super Intelligent Safety companies,	成立超级智能安全公司
886	what was your read on that situation?	你读到什么了?
887	So a lot of people who worked with Sam	所以很多人都和Sam合作过
888	said that maybe he's not the most direct person	说也许他不是最直接的人
889	in terms of being honest with them.	对他们诚实
890	And they had concerns about his views on safety.	他们担心他的安全观点
891	That's part of it.	这是它的一部分。
892	So they wanted more control,	所以他们想要更多的控制,
893	they wanted more concentration on safety.	他们想要更多关注安全
894	But also it seems that anyone who leaves that company	但似乎任何离开公司的人
895	and starts a new one gets a $20 billion valuation	然后开始一个新的 得到200亿美元的估值
896	just for having it started.	只是因为它开始。
897	You don't have a product, you don't have customers,	你没有产品,没有顾客,
898	but if you want to make many billions of dollars,	但如果你想赚几十亿美元
899	just do that.	就那样
900	So it seems like a very rational thing to do for anyone who can.	所以,这似乎是一个非常理性的事情 为任何人谁可以。
901	So I'm not surprised that there is a lot of attrition.	所以,我并不惊讶 有很多减员。
902	Meeting him in person, he's super nice, very smart,	亲自见他,他超棒,非常聪明
903	absolutely perfect public interface.	绝对完美的公共界面。
904	You see him testify in the Senate.	你看到他在参议院作证了
905	He says the right thing to the senators.	他对参议员说对了
906	You see him talk to the investors,	你看到他跟投资人说话
907	they get the right message.	他们得到正确的信息。
908	But if you look at what people who know him personally are saying,	但如果你看看认识他的人在说什么
909	it's probably not the right person to be	也许不是合适的人
910	controlling a project of that impact.	控制这个撞击的项目。
911	He puts safety second.	他把安全放在第二位。
912	Second to?	第二对?
913	Winning this race to super intelligence,	赢得超级智能的比赛
914	being the guy who created God,	是创造真主的,
915	and controlling the light cone of the universe,	控制宇宙的光锥,
916	he's worse.	更惨了
917	Do you suspect that's what he's driven by,	你怀疑他就是这个原因吗?
918	is by the legacy of being an impactful person	是一个有影响力的人的遗产
919	that did a remarkable thing versus	做了一件了不起的事,对
920	the consequence that that might have for society?	这可能给社会带来的后果?
921	Because it's interesting that his other startup is WorldCoin,	因为有趣的是,他的另一个启动 是WorldCoin,
922	which is basically a platform to create universal basic income,	基本上是一个创造普遍基本收入的平台,
923	i.e.	即说.
924	a platform to give us income in a world where	在这样一个世界里给我们收入的平台
925	people don't have jobs anymore.	人们已经没有工作了
926	So on one hand you're creating an AI company,	所以一方面你正在创建AI公司
927	on the other hand you're creating a company	另一方面 你正在创建一家公司
928	that is preparing for people not to have employment.	也就是为人们没有工作做准备。
929	It also has other properties.	它也有其他属性.
930	It keeps track of everyone's biometrics.	它能追踪每个人的生物特征
931	It keeps you in charge of the world's economy,	它让你负责世界经济
932	world's wealth.	世界财富.
933	They're retaining a large portion of WorldCoin.	他们保留了一大部分的世界币。
934	So I think it's kind of a very reasonable part	所以,我觉得这是一个非常合理的部分
935	to integrate with world dominance.	与世界支配地位相结合。
936	If you have a super intelligence system	如果你有超级情报系统
937	and you control money,	你控制金钱,
938	you're doing well.	你做得不错。
939	Why would someone want world dominance?	为什么有人想要主宰世界?
940	People have different levels of ambition.	人们有不同程度的野心.
941	Then you're a very young person with billions of dollars, fame.	你是一个非常年轻的 拥有数十亿美元,名声。
942	You start looking for more ambitious projects.	你开始寻找更雄心勃勃的项目。
943	Some people want to go to Mars.	有些人想去火星。
944	Others want to control the light cone of the universe.	其他人想要控制宇宙的光锥.
945	What did you say?	你说什么?
946	Light coin of the universe?	宇宙的光币?
947	Light cone.	轻圆锥.
948	So every part of the universe light can reach from this point,	因此,宇宙的每一个部分 光从这一点,
949	meaning anything accessible you want to grab	意思是任何您想要获取的可访问信息
950	and bring into your control.	并带入你的控制。
951	Do you think Sam Altman wants to control	你认为萨姆·阿尔特曼想控制
952	every part of the universe?	宇宙的每一个地方?
953	I suspect he might, yes.	我怀疑他会,是的。
954	It doesn't mean he doesn't want a side effect	不代表他不想有副作用
955	of it being a very beneficial technology,	它是一种非常有益的技术,
956	which makes all the humans happy.	这让所有人类都快乐
957	Happy humans are good for control.	快乐的人类善于控制.
958	If you had to guess what the world looks like in 2100,	如果你猜到2100年的世界是什么样子
959	if you had to guess?	如果你不得不猜测?
960	It's either free of human existence	要么没有人类的存在
961	or it's completely not comprehensible to someone like us.	或者我们这样的人根本无法理解
962	It's one of those extremes.	这是其中的一个极端。
963	So there's either no humans.	所以要么没有人类
964	It's basically the world is destroyed	基本上世界被毁灭了
965	or it's so different that I cannot envision those predictions.	我无法想象这些预言
966	What can be done to turn this ship	怎样才能让这艘飞船转弯
967	to a more certain positive outcome at this point?	此时此刻是否取得了更肯定的积极成果?
968	Is there still things that we can do or is it too late?	还有什么我们能做的 还是太迟了?
969	So I believe in personal self-interest.	所以我相信个人利益
970	If people realize that doing this thing	如果人们意识到这样做
971	is really bad for them personally,	对他们个人来说真的很不好
972	they will not do it.	他们不会这样做。
973	So our job is to convince everyone	所以我们的工作就是说服大家
974	with any power in this space,	与任何力量 在这个空间,
975	creating this technology, working for those companies.	创造这种技术,为那些公司工作。
976	They are doing something very bad for them.	他们正在为他们做一些非常坏的事情。
977	Not just forget other 8 billion people	不只是忘记其他80亿人
978	you're experimenting on with no permission,	你正在试验 没有允许,
979	no consent.	没有同意。
980	You will not be happy with the outcome.	你对结果不会满意的
981	If we can get everyone to understand that's a default,	如果我们能让所有人都明白这是默认
982	and it's not just me saying it,	不只是我说,
983	you had Geoff Hinton, Nobel Prize winner,	诺贝尔奖得主杰夫·欣顿
984	founder of the whole machine learning space.	整个机器学习空间的创始人.
985	He says the same thing.	他也是这么说的
986	Ben Geo, dozens of others, top scholars.	班杰尔,数十人,顶尖学者.
987	We had a statement about the interest of AI	我们有一份声明 有关AI的兴趣
988	signed by thousands of scholars, computer scientists.	几千名学者 计算机科学家签名
989	This is basically what we think right now	这基本上是我们现在的想法
990	and we need to make it a universal.	我们需要使它具有普遍性。
991	No one should disagree with this.	任何人都不应不同意这一点。
992	And then we may actually make good decisions	然后我们可能会做出正确的决定
993	about what technology to build.	关于什么技术可以建设。
994	It doesn't guarantee long-term safety for humanity,	它不能保证人类的长期安全
995	but it means we're not trying to get there	但这意味着我们没有试图到达那里
996	as soon as possible to the worst possible outcome.	尽快取得最糟糕的结果。
997	And are you hopeful that that's even possible?	你希望那有可能吗?
998	I want to try.	我想试试
999	We have no choice but to try.	我们别无选择,只能试试
1000	And what would need to happen and who would need to act?	需要发生什么,谁需要采取行动?
1001	Is it government legislation?	是政府立法吗?
1002	Is it?	是吗?
1003	Unfortunately, I don't think making it illegal is sufficient.	不幸的是,我不认为 使它非法是足够的。
1004	There are different jurisdictions.	有不同的管辖权。
1005	There is loopholes.	这里有漏洞。
1006	And what are you going to do if somebody does it?	如果有人这样做你会怎么做?
1007	You're going to fine them for destroying humanity?	你要罚他们破坏人类?
1008	Very steep fines for it?	罚款非常高?
1009	What are you going to do?	你打算怎么办?
1010	It's not enforceable.	不可执行.
1011	If they do create it, now the superintelligence is in charge.	如果他们真的创造了它,现在超级情报机构负责。
1012	So the judicial system we have is not impactful.	因此,我们的司法制度没有影响。
1013	And all the punishments we have are designed for punishing humans.	我们所有的惩罚都是为了惩罚人类
1014	Prison's capital punishment doesn't apply to AI.	监狱的死刑不适用于AI.
1015	You know, the problem I have is when I have these conversations,	你知道,我有的问题是 当我有这些谈话,
1016	I never feel like I walk away with a hope that something's going to go well.	我从不觉得自己走开 希望事情会顺利
1017	And what I mean by that is I never feel like I walk away with clear,	我的意思是,我从来没有感觉 我走得很清楚,
1018	some kind of clear set of actions that can course correct what might happen here.	一种明确的行动 能够纠正这里可能发生的事情
1019	So what should I do?	那我该怎么办?
1020	What should the person sat at home listening to this do?	坐在家里的人应该听这个做什么?
1021	You talk to a lot of people who are building this technology.	你和许多正在建立这种技术的人交谈。
1022	Ask them precisely to explain some of those things they claim to be impossible,	请他们准确地解释一些 他们声称不可能的事情,
1023	how they solved it or going to solve it before they get to where they're going.	他们是如何解决的 或打算解决 它之前,他们到达 他们去的地方。
1024	Do you know, I don't think Sam Altman wants to talk to me.	你知道,我不认为 Sam Altman 想跟我说话。
1025	I don't know.	师曰.
1026	He seems to go on a lot of podcasts.	他好像经常播客
1027	Maybe he does.	也许他有
1028	He wants to go online.	他想上网
1029	I wonder why that is.	我想知道为什么
1030	I'd love to speak to him, but I don't think he wants me to interview him.	我很想和他谈谈 但我不认为他想让我采访他
1031	Have an open challenge.	有公开的挑战。
1032	Maybe money is not the incentive, but whatever attracts people like that,	也许金钱不是刺激 但无论吸引到什么人
1033	whoever can convince you that it's possible to control and make safe super intelligence,	谁能说服你 有可能控制并制造安全的超级情报
1034	gets the prize, they come on your show and prove their case.	拿到奖项,他们来你的节目 证明他们的情况。
1035	Anyone.	谁都行
1036	If no one claims the price or even accepts the challenge after a few years,	如果几年后没人要求价格甚至接受挑战
1037	maybe we don't have anyone with solutions.	也许我们没有解决的办法
1038	We have companies valued again at billions and billions of dollars	我们公司又值数十亿
1039	working on safe super intelligence.	研究安全超级情报
1040	We haven't seen their output yet.	我们还没有看到他们的输出。
1041	Yeah, I'd like to speak to Ilya as well,	我也想跟伊利亚谈谈
1042	because I know he's working on safe super intelligence.	因为我知道他在研究安全的超级情报
1043	Notice a pattern too.	注意一个模式。
1044	If you look at history of AI safety organizations	如果你看看AI安全组织的历史
1045	or departments within companies, they usually start well, very ambitious,	或公司内部的部门, 他们通常开始良好, 非常雄心勃勃,
1046	and then they fail and disappear.	然后他们失败并消失
1047	So OpenAI had super intelligence alignment team.	因此OpenAI拥有超级情报协调团队.
1048	The day they announced it, I think they said they're going to solve it in four years,	他们宣布的那天,我想他们说 他们将在四年内解决它,
1049	but half a year later, they canceled the team.	但半年后,他们取消了球队。
1050	And there is dozens of similar examples.	还有几十个类似的例子。
1051	Creating a perfect safety for super intelligence,	为超级情报创造完美的安全
1052	perpetual safety as it keeps improving, modifying, interacting with people.	永远的安全,因为它不断改善,修改, 与人互动。
1053	You're never going to get there.	你永远不会到达那里。
1054	It's impossible.	这是不可能的。
1055	There is a big difference between difficult problems in computer science	计算机科学的难题大不相同
1056	and P complete problems and impossible problems.	问题和不可能的问题
1057	And I think control, indefinite control of super intelligence is such a problem.	我认为控制, 无限期控制超级情报是这样一个问题。
1058	So what's the point trying then if it's impossible?	那么,如果不可能的话,有什么意义呢?
1059	Well, I'm trying to prove that it is specifically that.	嗯,我试图证明 它是特别的。
1060	Once we establish something is impossible,	一旦我们找到不可能的东西
1061	fewer people will waste their time claiming they can do it	少有人会浪费时间 声称他们能做到
1062	and find looking for money.	找找钱
1063	So many people go and give me a billion dollars in two years	这么多人在两年内给我十亿
1064	and I'll solve it for you.	我会帮你解决的
1065	Well, I don't think you will.	我想你不会的
1066	But people aren't going to stop striving towards it.	但人们是不会停止努力的。
1067	So if there's no attempts to make it safe	所以,如果没有尝试 使它安全
1068	and there's more people increasingly striving towards it, then it's inevitable.	而越来越多的人 越来越努力去争取它, 那么这是不可避免的。
1069	But it changes what we do.	但它改变了我们的工作。
1070	If we know that it's impossible to make it right, to make it safe,	如果我们知道这是不可能 使其正确,使之安全,
1071	then this direct path of just build it as soon as you can becomes suicide mission.	然后这个直接的道路,只要你能够 成为自杀任务。
1072	Hopefully fewer people will pursue that.	希望很少有人会追求这一点。
1073	They may go in other directions.	他们可能往其他方向走。
1074	Like, again, I'm a scientist, I'm an engineer.	就像,再次,我是科学家,我是工程师。
1075	我爱AI。
1076	I love technology.	我喜欢科技
1077	I use it all the time.	我时常使用.
1078	Build useful tools.	建立有用的工具。
1079	Stop building agents.	停止建立特工。
1080	Build narrow super intelligence, not a general one.	建立狭隘的超级智能,而不是一般的智能.
1081	I'm not saying you shouldn't make billions of dollars.	我不是说你不应该赚几十亿美元
1082	I love billions of dollars.	我爱数十亿美金。
1083	But don't kill everyone, yourself included.	但别杀光所有人 包括你自己
1084	They don't think they're going to, though.	他们认为他们不会这么做
1085	Then tell us why.	那就告诉我们原因
1086	I hear things about intuition.	我听到一些关于直觉的事
1087	I hear things about we'll solve it later.	我听说我们以后会解决的
1088	Tell me specifically in scientific terms.	具体用科学术语告诉我.
1089	Publish a peer-reviewed paper explaining how you're going to control super intelligence.	发表一份同行评审论文,解释如何控制超级智能.
1090	Yeah, it's strange.	是啊,这是奇怪的。
1091	It's strange to even bother if there was even a 1% chance of human extinction.	连人类灭绝的几率都只有1% 也奇怪
1092	It's strange to do something.	作么生是甚么怪处.
1093	Like, if there was a 1% chance.	比如,如果有1%的机会。
1094	Someone told me there was a 1% chance that if I got in a car, I might not be alive.	有人告诉我,有1%的机率 如果我上车,我可能还活着。
1095	I would not get in the car.	我不会上车的
1096	If you told me there was a 1% chance that if I drank whatever liquid is in this cup right now,	如果你告诉我有1%的机会 如果我喝掉这杯子里的液体
1097	I might die, I would not drink the liquid.	我可能会死,我不会喝液体。
1098	Even if there was a billion dollars, if I survived.	即使有十亿美元, 如果我活下来。
1099	So the 99% chance that I get a billion dollars, the 1% is I wouldn't drink it.	所以,99%的机会,我得到10亿美元, 1%是我不会喝它。
1100	I wouldn't take the chance.	我不会冒这个险的
1101	It's worse than that.	更无比也.
1102	Not just you die.	不仅仅是你死了。
1103	Everyone dies.	每个人都会死
1104	Yeah, yeah.	对 对
1105	Now, would we let you drink it at any odds?	现在,我们会让你喝它 在任何机会?
1106	That's for us to decide.	由我们决定
1107	You don't get to make the choice for us to get consent from human subjects.	你不能让我们选择 得到人类主体的同意
1108	You need them to comprehend what they are consenting to.	你需要他们明白他们同意什么
1109	If those systems are unexplainable, unpredictable, how can they consent?	如果这些系统是无法解释的、无法预测的,它们怎么能同意呢?
1110	They don't know what they are consenting to.	他们不知道他们同意什么
1111	So it's impossible to get consent by definition.	所以从定义上说不可能获得同意.
1112	So this experiment can never be run ethically.	所以这个实验永远不能道德地进行.
1113	By definition, they are doing an ethical experimentation on human subjects.	顾名思义,他们正在对人类课题进行道德实验。
1114	Do you think people should be protesting?	你觉得人们应该抗议吗?
1115	There are people protesting.	有人抗议。
1116	There is stop AI, there is pause AI, they block offices of open AI, they do it weekly,	有停止AI,有暂停AI, 他们封锁办公室开放AI, 他们每周做,
1117	monthly, quite a few actions and they're recruiting new people.	每月,很多行动 他们招募新人。
1118	Do you think more people should be protesting?	你觉得应该有更多的人抗议吗?
1119	Do you think that's an effective solution?	你认为这是有效的解决办法吗?
1120	If you can get it to a large enough scale to where majority of population is participating,	如果你能把它 达到足够大的规模 到大多数人参与,
1121	it would be impactful.	这将是影响很大。
1122	I don't know if they can scale from current numbers to that,	我不知道他们能不能从现在的数字到那个
1123	but I support everyone trying everything peacefully and legally.	但我支持每个人 尝试一切和平和合法。
1124	And for the person listening at home, what should they be doing?	对于在家听话的人, 他们应该做什么?
1125	Because they don't want to feel powerless.	因为他们不想感到无能为力。
1126	None of us want to feel powerless.	没有人想感到无能为力。
1127	So it depends on what scale we're asking about time scale.	这取决于我们要求的时间尺度。
1128	I was saying like this year your kid goes to college, what major to pick?	我说你孩子今年上大学 选什么专业?
1129	Should they go to college at all?	他们应该上大学吗?
1130	Yeah.	对
1131	Should you switch jobs?	你该换工作吗?
1132	Should you go into certain industries?	你应该进入某些行业吗?
1133	Those questions we can answer.	这些问题我们可以回答。
1134	We can talk about immediate future.	我们可以谈论近期的未来。
1135	What should you do in five years with this being created?	五年后你应该怎么做?
1136	For an average person, not much.	对普通人来说,没什么
1137	Just like they can't influence World War III, nuclear, Holocaust, anything like that.	就像他们不能影响第三次世界大战,核战争,大屠杀,任何类似的事情一样.
1138	It's not something anyone's going to ask them about.	这不是别人要问的
1139	Today, if you want to be a part of this movement, yeah, join POS.AI, join STOP.AI.	今天,如果你想参与这个运动, 是的,加入POS.AI,加入STOP.AI。
1140	Those organizations currently trying to build up momentum to	这些组织目前正在努力建立发展势头,以便:
1141	bring democratic powers to influence those individuals.	使民主力量影响这些人。
1142	So in the near term, not a huge amount.	因此,在近期内,不是很多。
1143	I was wondering if there are any interesting strategies in the near term.	我在想,近期内是否有任何有趣的战略。
1144	Should I be thinking differently about my family?	我应该对我的家人有不同的想法吗?
1145	I mean, you've got kids, right?	我的意思是,你有孩子,对不对?
1146	You've got three kids?	你有三个孩子?
1147	That I know about, yeah.	我知道,没错。
1148	How are you thinking about parenting in this world that you see around the corner?	你在想什么 在这个你看到的角落里
1149	How are you thinking about what to say to them,	你在想什么 告诉他们,
1150	the advice to give them what they should be learning?	教他们怎样学习?
1151	So there is general advice outside of this domain that you should live your	因此,在这个领域之外,有一些一般的建议是,你应该活在你的
1152	every day as if it's your last.	每天都像你最后一样
1153	It's a good advice no matter what.	不管怎样,都是个好建议
1154	If you have three years left or 30 years left, you lived your best life.	如果你还有3年或30年, 你的生活是最好的。
1155	So try to not do things you hate for too long.	所以别做你讨厌的事太久了
1156	Do interesting things.	做点有趣的事
1157	Do impactful things.	做一些有影响的事
1158	If you can do all that while helping people do that.	如果你能在帮助别人的同时做这些事
1159	Simulation theory is an interesting sort of adjacent subject here because	模拟理论是一个有趣的相邻主题 因为
1160	as computers begin to accelerate and get more intelligent and we're able to	随着计算机开始加速 并获得更多的智能 我们有能力
1161	do things with AI that we could never have imagined in terms of like	做一些我们无法想象的事情
1162	imagine the worlds that we could create with virtual reality.	想象我们用虚拟现实创造的世界。
1163	I think it was Google that recently released, what was it called?	我认为是谷歌最近发布的, 它叫什么?
1164	Like the AI worlds.	犹如AI世界.
1165	You take a picture and it generates a whole world.	你照张相就能创造整个世界
1166	Yeah, you can move through the world.	是啊,你可以移动 通过世界。
1167	I'll put it on the screen for people to see but Google have released this technology	我把它放在屏幕上让人们看到 但谷歌已经发布了这个技术
1168	which allows you, I think with a simple prompt actually, to make a three-dimensional world	它允许你,我认为 一个简单的提示,实际上, 一个三维世界
1169	that you can then navigate through and in that world it has memory.	你能够航行过这个世界,它有记忆。
1170	So in the world if you paint on a wall and turn away, you look back the wall.	所以在世界上,如果你在墙上画画,然后转过身去,你就回头看墙。
1171	It's persistent.	执着.
1172	Yeah, it's persistent.	是啊,这是持久的。
1173	And when I saw that, I go, God, Jesus, bloody hell, this is like the	当我看到,我走了,上帝,耶稣,该死的地狱, 这就像
1174	hills of being able to create a simulation that's indistinguishable from everything I see here.	山丘能够创造出一个模拟 与我在这里看到的一切是分不开的。
1175	Right.	对
1176	That's why I think we are in one.	这就是为什么我认为我们在一个。
1177	That's exactly the reason.	这正是原因
1178	AI is getting to the level of creating human agents, human level agents,	AI正在进入 创建人类代理,人级代理,
1179	and virtual reality is getting to the level of being indistinguishable from ours.	虚拟现实正变得与我们无法区分
1180	So you think this is a simulation?	你觉得这是模拟吗?
1181	I'm pretty sure we are in a simulation, yeah.	我敢肯定,我们是在模拟,是的。
1182	For someone that isn't familiar with the simulation arguments,	对于不熟悉模拟辩论的人来说,
1183	what are the first principles here that convince you that we are currently	在这里,什么是第一个原则 说服你,我们现在
1184	living in a simulation?	住在模拟中?
1185	So you need certain technologies to make it happen.	所以,你需要某些技术来实现它。
1186	If you believe we can create human level AI,	如果你相信我们能创造出人类层面的AI
1187	and you believe we can create virtual reality as good as this in terms of resolution,	你相信我们可以创造出 虚拟现实,就像在分辨率方面,
1188	haptics, whatever properties it has,	不幸,无论它有什么属性,
1189	then I commit right now the moment this is affordable,	然后,我现在就承诺 此刻,这是负担得起的,
1190	I'm going to run billions of simulations in this exact moment,	我将在这一刻进行数十亿的模拟
1191	making sure you are statistically in one.	确保您在统计上在一个。
1192	Say that last part again, you're going to run?	再说一遍,你要跑吗?
1193	I'm going to commit right now when it's very affordable,	我现在要承诺 当它非常负担得起,
1194	it's like 10 bucks a month to run it.	这就像10块钱一个月运行。
1195	I'm going to run a billion simulations of this interview.	我要对这次采访进行十亿次模拟
1196	Why?	为什么?
1197	Because statistically that means you are in one right now.	因为从统计学上讲,这意味着你现在是一个人。
1198	The chances of you being in the real one is one in a billion.	你进入现实的机会是十亿分之一
1199	Okay, so to make sure I'm clear on this.	好吧,所以确定 我清楚这一点。
1200	It's a retroactive placement.	追溯安置.
1201	Yeah, so the minute it's affordable,	是啊,所以一旦它可以买得起,
1202	then you can run billions of them,	然后你可以运行数十亿,
1203	and they would feel and appear to be exactly like this interview right now.	他们现在会觉得和看起来 和这次采访完全一样
1204	So assuming the AI has internal states experiences qualia,	所以假设AI有 内部状态经历夸利亚,
1205	some people argue that they don't, some say they already have it.	有些人认为他们没有,有些人说他们已经有了。
1206	That's a separate philosophical question.	这是一个单独的哲学问题。
1207	But if we can simulate this, I will.	但是如果我们可以模拟,我会的。
1208	Some people might misunderstand.	有些人可能会误会
1209	You're not saying that you will.	你不是说你会的
1210	You're saying that someone will.	你是说有人会的
1211	I can also do it.	我也能做到这一点。
1212	I don't mind.	没关系
1213	Of course others will do it before I get there.	我到之前当然会有人这么做
1214	If I'm getting it for $10, somebody got it for $1000.	如果我要10块 有人要1000块
1215	That's not the point.	这不是重点。
1216	If you have technology, we're definitely running a lot of simulations	如果你有技术,我们肯定在做很多模拟
1217	for research, for entertainment, games, all sorts of reasons.	研究 娱乐 游戏 各种原因
1218	And the number of those greatly exceeds the number of real worlds we're in.	这些数量大大超过我们所处现实世界的数量。
1219	Look at all the video games kids are playing.	看看孩子们在玩的电子游戏
1220	Every kid plays 10 different games.	每个孩子都玩10个不同的游戏.
1221	You know, billion kids in the world.	世界上有十亿个孩子
1222	So there is 10 billion simulations in one real world.	因此在一个真实的世界里有100亿的模拟。
1223	Even more so when we think about advanced AI, super intelligent systems.	当我们想到先进的AI,超级智能系统时,更是这样.
1224	Their thinking is not like ours.	他们的想法跟我们不一样
1225	They think in a lot more detail.	他们想的更多细节。
1226	They run experiments.	他们运行实验。
1227	So running a detailed simulation of some problem at the level of creating artificial humans	在创造人造人类的层面上 对一些问题进行详细的模拟
1228	and simulating the whole planet would be something they'll do routinely.	模拟整个星球是他们例行公事
1229	So there is a good chance this is not me doing it for $10.	因此,这很有可能不是我10美元。
1230	It's a future simulation thinking about something in this world.	这是一个未来的模拟 思考在这个世界上的东西。
1231	So it could be the case that...	所以有可能是这样...
1232	So it could be the case that....	那样的话...

该视频共有字幕 1233 条。解锁更多字幕为会员功能，请移动到 价格

P2 · 【AI安全】AI安全专家：2030年仅存的5类工作 · 未来职业预测 p02 part 2 (P2)

【AI安全】AI安全专家：2030年仅存的5类工作 · 未来职业预测 p02 part 2 (P2)

发布于 2026-08-13 · 时长 27:25

字幕摘录

时间	英文	中文
0:00	where they could affordably run simulations that are indistinguishable from this and they decided to do it and this is it right now.	在那里,他们可以承受得起 模拟,这是无法区分的 他们决定这样做,这是现在。
0:12	And it would make sense that they would run simulations as experiments or for games or for entertainment.	并且,他们会进行模拟,作为实验, 游戏或娱乐。
0:17	And also, when we think about time in the world that I'm in, in this simulation that I could be in right now, time feels long, relatively.	而且,当我们思考我所处世界的时间时, 在这种模拟中, 时间感觉很长,相当长。
0:24	You know, I have 24 hours in a day, but in their world, it could be...	你知道,我一天有24小时, 但在他们的世界,它可能是...
0:30	Time is relative.	时间是相对的。
0:31	Relative, yeah, it could be a second.	相对的,是的,它可能是第二。
0:32	My whole life could be a millisecond in there.	我的整个生活可能在那里一毫秒。
0:35	Right, you can change the speed of simulations you're running for sure.	是的,你可以改变模拟的速度 你运行的确定。
0:40	So your belief is that this is probably a simulation?	所以你相信这可能是个模拟?
0:43	Most likely.	可能吧 可能吧 可能吧 可能吧

展开字幕全文（469 条）

序号	英文	中文
1	where they could affordably run simulations that are indistinguishable from this and they decided to do it and this is it right now.	在那里,他们可以承受得起 模拟,这是无法区分的 他们决定这样做,这是现在。
2	And it would make sense that they would run simulations as experiments or for games or for entertainment.	并且,他们会进行模拟,作为实验, 游戏或娱乐。
3	And also, when we think about time in the world that I'm in, in this simulation that I could be in right now, time feels long, relatively.	而且,当我们思考我所处世界的时间时, 在这种模拟中, 时间感觉很长,相当长。
4	You know, I have 24 hours in a day, but in their world, it could be...	你知道,我一天有24小时, 但在他们的世界,它可能是...
5	Time is relative.	时间是相对的。
6	Relative, yeah, it could be a second.	相对的,是的,它可能是第二。
7	My whole life could be a millisecond in there.	我的整个生活可能在那里一毫秒。
8	Right, you can change the speed of simulations you're running for sure.	是的,你可以改变模拟的速度 你运行的确定。
9	So your belief is that this is probably a simulation?	所以你相信这可能是个模拟?
10	Most likely.	可能吧 可能吧 可能吧 可能吧
11	And there is a lot of agreement on that.	关于这一点,有许多一致意见。
12	If you look again, returning to religions, every religion basically describes what?	如果你再看一遍,回到宗教,每个宗教基本上都描述了什么?
13	A super intelligent being, an engineer, a programmer creating a fake world for testing purposes or for whatever.	一个超级聪明的人,一个工程师,一个程序员 创造出一个假的世界 用于测试目的或者什么的。
14	But if you took the simulation hypothesis paper, you go to jungle, you talk to primitive people, a local tribe, and in their language, you tell them about it.	但如果你拿模拟假说纸,你去丛林, 和原始人,一个当地部落交谈, 用他们的语言,你告诉他们。
15	Go back two generations later.	后两代后归.
16	They have religion.	他们有宗教。
17	That's basically what the story is.	基本上就是这个故事
18	Religion, yeah, it describes a simulation theory.	宗教,是的,它描述了一个模拟理论。
19	Somebody created it.	有人创造了它。
20	By default, that was the first theory we had.	默认,这是我们的第一个理论。
21	And now with science, more and more people are going like, I'm giving it non-trivial probability.	现在随着科学的发展, 越来越多的人会像, 我给它非三角概率。
22	A few people are as high as I am, but a lot of people give it some credence.	有几个人跟我一样高, 但很多人给予它一些信任。
23	What percentage are you at in terms of believing that we are currently living in a simulation?	你相信我们目前生活在模拟中的百分比是多少?
24	Very close to certainty.	非常接近于确定性。
25	And what does that mean for the nature of your life?	这对你生命的本质意味着什么?
26	If you're close to 100% certain that we are currently living in a simulation, does that change anything in your life?	如果你几乎百分之百确定 我们目前生活在模拟中, 这是否改变了你的生活?
27	So all the things you care about are still the same.	所以,所有你关心的事情 仍然是相同的。
28	Pain still hurts, love still love, right?	痛还是痛,爱还是爱,对不对?
29	Like those things are not different, so it doesn't matter.	喜欢那些东西并不不同,所以无所谓.
30	They're still important.	他们仍然很重要。
31	That's what matters.	这才是最重要的。
32	The little 1% difference is that I care about what's outside the simulation.	1%的差别在于我关心模拟之外的东西。
33	I want to learn about it.	欲学诸法.
34	I write papers about it.	我写关于它的论文。
35	So that's the only impact.	这就是唯一的影响。
36	And what do you think is outside of the simulation?	你觉得模拟之外有什么?
37	I don't know, but we can look at this world and derive some properties of the simulators.	我不知道,但我们可以看看这个世界 并得出模拟器的一些特性。
38	So clearly, brilliant engineer, brilliant scientist, brilliant artist.	这么清楚,杰出的工程师,杰出的科学家,杰出的艺术家。
39	Not so good with morals and ethics.	不善于道德和伦理
40	Room for improvement.	改进的空间。
41	In our view of what morals and ethics should be.	在我们看来,什么是道德和伦理。
42	Well, we know there is suffering in the world.	我们知道世界上有苦难
43	So unless you think it's ethical to torture children, then I'm questioning your approach.	所以除非你认为折磨孩子是合乎道德的, 那我质疑你的做法。
44	But in terms of incentives to create a positive incentive, you probably also need to create negative incentives.	但对于创造积极激励的激励,你可能需要创造消极激励.
45	Suffering seems to be one of the negatives and incentives built into our design to stop me doing things I shouldn't do.	痛苦似乎是我们设计中的消极因素和激励因素之一 用来阻止我做我不该做的事
46	So like put my hand in a fire, it's going to hurt.	所以,像把我的手在火中, 它会伤害。
47	But it's all about levels, levels of suffering, right?	但一切都是关乎痛苦的程度,对吧?
48	So unpleasant stimuli, negative feedback doesn't have to be at like negative infinity hell levels.	所以令人不快的刺激,负面的反馈不必像负面的无限地狱水平.
49	You don't want to burn a life and feel it.	不欲烧一命而觉.
50	You want to be like, oh, this is uncomfortable.	你想成为,哦,这是不舒服的。
51	I'm going to stop.	我要停下来。
52	It's interesting because we assume that they don't have great morals and ethics.	这很有趣,因为我们假设 他们没有伟大的道德和伦理。
53	But we too would take animals and cook them and eat them for dinner.	但我们也会带动物去煮饭吃
54	And we also conduct experiments on mice and rats.	我们还对小鼠和小鼠进行实验。
55	But to get university approval to conduct an experiment, you submit a proposal.	但是,为了得到大学的批准来进行实验,你提出一个建议.
56	And there is a panel of ethicists who would say, you can't experiment on humans.	还有一群伦理学家说,你不能在人类身上做实验。
57	You can't burn babies.	你不能烧婴儿。
58	You can't eat animals alive.	不能生食畜生.
59	All those things will be banned in most parts of the world where they have ethical boards.	在世界上大多数有道德板的地方,所有这些东西都将被禁止。
60	Yeah.	对
61	Some places don't bother with it.	有的地方不烦.
62	So they have easier approval process.	所以他们的审批程序比较简单.
63	It's funny when you talk about the simulation theory, there's an element of the conversation that makes life feel less meaningful in a weird way.	当你谈论模拟理论时,很有趣的是,对话中有一个元素,以怪异的方式使生活感觉不太有意义.
64	I know it doesn't matter.	我知道这不重要
65	But whenever I have this conversation with people, not on the podcast about are we living in a simulation,	但每当我和别人谈话时 播客不会说我们生活在模拟中
66	you almost see a little bit of meaning come out of their life for a second.	你几乎看到了一点意义 从他们的生活中出来一秒钟。
67	And then they forget and then they carry on.	然后他们忘记 然后继续
68	But the thought that this is a simulation almost posits that it's not important.	但认为这是一个模拟 几乎假设它不重要。
69	Or that I think humans want to believe that this is the highest level and we're at the most important and we're the it's all about us.	或者说,我认为人类想要相信 这是最高的层次,我们处于最重要的,我们才是最重要的。
70	We're quite egotistical by design.	我们从设计上来说很自私
71	And just an interesting observation I've always had when I have these conversations with people that it seems to strip something out of their life.	一个有趣的观察,我总是有 当我有这些谈话 与人们,它似乎剥离的东西 他们的生活。
72	Do you feel religious people feel that way?	你觉得宗教人士有这种感觉吗?
73	They know there is another world and the one that matters is not this one.	他们知道还有另一个世界,而这个世界并不重要。
74	Do you feel they don't value their lives the same?	你觉得他们不珍惜同样的生命吗?
75	I guess in some religions.	我想在某些宗教里
76	I think they think that this world is being created for them and that they are going to go to this heaven or hell.	我认为,他们认为,这个世界是为他们创造的,他们将进入这个天堂或地狱。
77	And that still puts them at the very centre of it.	这使得它们仍然处于核心。
78	But if it's a simulation, we could just be some computer game that a four year old alien is messing around with while he's got some time to burn.	但如果是模拟的话, 我们可以做一个电脑游戏, 一个四岁的外星人趁他有时间燃烧时, 正在乱搞。
79	But maybe there is a test and there is a better simulation you go to and a worse one.	但也许有一个测试,有一个更好的模拟 你要去和更糟。
80	Maybe there are different difficulty levels.	也许有不同的难度水平。
81	Maybe you want to play it on a harder setting next time.	也许下次你想在更难的场合玩
82	I've just invested millions into this and become a co-owner of the company.	我刚刚投入了上百万 成为了公司的共同所有者
83	是一家叫Ketone IQ的公司.
84	And the story is quite interesting.	这个故事相当有趣。
85	I started talking about ketosis on this podcast and the fact that I'm very low carb, very, very low sugar, and my body produces ketones, which have made me incredibly focused, have improved my endurance, have improved my mood and have made me more capable	我开始谈论这个播客的酮化, 以及我非常低的碳化物, 非常,非常低的糖, 我的身体生产酮, 这使我变得非常集中, 改善了我的耐力, 改善了我的心情, 并且让我更有能力做我在这里做的事。
86	And because I was talking about it on the podcast, a couple of weeks later, these showed up on my desk in my HQ in London, these little shots.	因为几周后我在播客上谈到这个, 这些出现在我在伦敦总部的桌子上, 这些小镜头。
87	And oh my God, the impact this had on my ability to articulate myself, on my focus, on my workouts, on my mood, on stopping me crashing throughout the day was so profound that I reached out to the founders of the company.	哦,我的上帝,这对我表达自己的能力, 我的焦点,我的锻炼,我的心情, 阻止我坠机 整天如此深刻,我联系了公司的创始人。
88	And now I'm a co-owner of this business.	现在我是这行的共同主人
89	I highly, highly recommend you look into this.	我高度,高度建议你调查一下
90	I highly recommend you look at the science behind the product.	我强烈建议你看看产品背后的科学
91	如果你想自己试试,请访问ketone.com slash Stephen 以30%的订阅订单。
92	And you'll also get a free gift with your second shipment.	还有你的第二批货 也可以免费买礼物
93	这是ketone.com 斜线斯蒂芬。
94	And I'm so honoured that once again, a company I own can sponsor my podcast.	我非常荣幸,我拥有的公司 可以再次赞助我的播客。
95	I've built companies from scratch and backed many more.	我从零开始建立公司 并且支持更多的公司
96	And there's a blind spot that I keep seeing in early stage founders.	还有一个盲点,我在早期一直看到创始人。
97	They spend very little time thinking about HR.	他们很少花时间去思考HR。
98	And it's not because they're reckless or they don't care.	且道不是因为他们鲁莽或不在乎.
99	It's because they're obsessed with building their companies.	这是因为他们沉迷于建设自己的公司.
100	And I can't fault them for that.	我不能怪他们
101	At that stage, you're thinking about the product, how to attract new customers, how to grow your team, really how to survive.	在那个阶段,你正在思考产品,如何吸引新的顾客,如何培养你的团队,真正如何生存.
102	And HR slips down the list because it doesn't feel urgent, but sooner or later it is.	HR从清单中滑下来 因为它不觉得紧急, 但迟早是。
103	And when things get messy, tools like our sponsor today, Justworks, go from being a nice to have to being a necessity.	当事情变得混乱, 工具像我们的赞助者今天, Justworks, 从一个好 必须成为需要。
104	Something goes sideways and you find yourself having conversations you did not see coming.	有些东西走边走边,你发现自己 对话,你没有看到来。
105	This is when you learn that HR really is the infrastructure of your company.	当你知道HR真的是你公司的基础设施时
106	And without it, things wobble.	没有它, 事情摇摆。
107	And Justworks stops you learning this the hard way.	并且只是工作停止你 学习这艰难的方式。
108	It takes care of the stuff that would otherwise drain your energy and your time, automating payroll, health insurance benefits.	它照顾那些会消耗你精力和时间的东西, 自动化的薪水,健康保险福利。
109	And it gives your team human support at any hour.	它为你们团队随时提供人力支援
110	It grows with your small business from start-up through to growth, even when you start hiring team members abroad.	它随着你的小企业从开办到成长而增长,即使你开始在国外招聘团队成员.
111	So if you want HR support that's there through the exciting times and the challenging times, head to Justworks.com now.	所以,如果你想要HR的支持, 这是经过 令人兴奋的时代和挑战的时代, 现在去Justworks.com。
112	That's Justworks.com.	这是Justworks.com. 互联网档案馆的存檔,存档日期2013-03-02.
113	And do you think much about longevity?	你对长寿有什么想法?
114	A lot, yeah.	很多,没错。
115	It's probably the second most important problem, because if AI doesn't get us, that will.	这也许是第二个最重要的问题, 因为如果AI不给我们,那会。
116	What do you mean?	你说什么?
117	You're going to die of old age.	年老将死.
118	Which is fine.	没关系
119	That's not good.	这可不好
120	You want to die?	你想死吗?
121	I mean...	我是说...
122	You don't have to.	你不必这样
123	It's just a disease.	谓病而已.
124	We can cure it.	我们可以治好它。
125	Nothing stops you from living forever.	没有什么能阻止你永远活着
126	Not as long as the universe exists, unless we escape the simulation.	除非我们逃脱模拟
127	But we wouldn't want a world where everybody could live forever, right?	但我们不希望一个每个人都能永生的世界,对吧?
128	That would be...	这将是...
129	Sure we do.	当然
130	Why?	为什么?
131	Who do you want to die?	你想杀谁?
132	I don't know.	师曰.
133	I mean, I say this because it's all I've ever known, that people die, but wouldn't the world become pretty overcrowded if...	我的意思是,我这样说是因为我所知道的一切, 人们死亡, 但世界不会变得相当拥挤,如果...
134	No, you stop reproducing if you live forever.	不,你停止复制 如果你永远活着。
135	You have kids because you want a replacement for you.	你有孩子是因为你想换人
136	If you live forever, you're like, I'll have kids in a million years.	如果你永远活着,你喜欢, 我将在一百万年后有孩子。
137	That's cool.	这很酷。
138	I'll go explore the universe first.	我会先去探索宇宙
139	Plus, if you look at actual population dynamics outside of like one continent, we're all shrinking.	此外,如果你看看一个大陆以外的人口动态, 我们都在缩小。
140	We're not growing.	我们没有成长。
141	Yeah, this is crazy.	是啊,这是疯了。
142	It's crazy that the more rich people get, the less kids they have, which aligns with what you're saying.	这太疯狂了,因为有钱人越多,孩子就越少,这与你说的一致.
143	And I do actually think, I think if, I'm going to be completely honest here, I think if I knew that I was going to live to a thousand years old, there's no way I'd be having kids at 30.	我确实认为,如果,如果,我要 完全诚实在这里, 我想如果我知道 我要活到一千岁,
144	All right, exactly.	好吧,没错。
145	Biological clocks are based on terminal points.	生物钟基于终端点.
146	Whereas if your biological clock is infinite, you'd be like, one day.	如果你的生物时钟是无限的, 你会喜欢,有一天。
147	And you think that's close, being able to extend our lives?	你觉得能延长我们的生活很近吗?
148	It's one breakthrough away.	这是一个突破。
149	I think somewhere in our genome, we have this rejuvenation loop and it's said to basically give us at most 120.	我认为,在基因组的某个地方, 我们有这个复兴循环, 据说它基本上给我们最多120。
150	I think we can reset it to something bigger.	我想我们可以把它重装到更大的东西。
151	AI is probably going to accelerate that.	AI可能要加速了。
152	That's one very important application area.	这是一个非常重要的应用区域。
153	Yes, absolutely.	是的,当然。
154	也许Brian Johnson说的对 别死
155	He keeps saying to me, he's like, don't die now.	他一直对我说,他像,不要死现在。
156	Don't die ever.	永远不要死
157	But he's saying like, don't die before we get to the technology.	但他说,不要死 在我们得到技术之前。
158	Right.	对
159	Longevity escape velocity.	长寿逃逸速度.
160	You want to live long enough to live forever.	欲长寿长寿长寿.
161	If at some point we, every year of your existence, add two years to your existence through medical breakthroughs, then you'll live forever.	如果在某个时候,我们,你存在的每一年, 通过医疗突破,给你的存在增加两年, 那么,你将永远活着。
162	You just have to make it to that point of longevity escape velocity.	你只需要让它到长寿点 逃脱速度。
163	And he thinks that longevity escape velocity, especially in the world of AI, is decades away minimum.	他认为长寿逃逸速度,特别是在AI世界,是几十年之外的最低点.
164	As soon as we fully understand human genome, I think we'll make amazing breakthroughs very quickly.	一旦我们完全了解人类基因组, 我想我们会很快地取得惊人的突破。
165	Because we know some people have genes for living way longer.	因为我们知道有些人的基因可以活得更长。
166	They have generations of people who are centenarians.	他们有几代人都是百年人
167	So if we can understand that and copy that, or copy it from some animals which will live forever, we'll get there.	所以,如果我们能理解,复制, 或者从一些动物那里复制, 它们会永远存在, 我们会到达那里。
168	Would you want to live forever?	你想永远活着吗?
169	Of course.	当然
170	Reverse the question.	倒置问题。
171	Let's say we lived forever and you ask me, do you want to die in 40 years?	假设我们永远活着 你问我,你想死40年吗?
172	Why would I say yes?	我为什么要答应?
173	I don't know.	师曰.
174	Maybe you're just used to the default.	也许你只是习惯了默认。
175	Yeah, I am used to the default.	是的,我已经习惯了默认。
176	And nobody wants to die.	没有人想死
177	Like no matter how old you are, nobody goes, yeah, I want to die this year.	无论你多大,没有人去, 是的,我想死今年。
178	Everyone's like, I want to keep living.	每个人都喜欢,我想继续生活。
179	I wonder if life and everything would be less special if I lived for 10,000 years.	我在想 如果我活了一万年 生活和一切都会变得不那么特殊
180	I wonder if going to Hawaii for the first time or, I don't know, a relationship, all of these things would be way less special to me if they were less scarce.	我在想,如果第一次去夏威夷 或者,我不知道, 一种关系,所有这些事情 如果它们不那么稀疏, 对我来说就不那么特别了。
181	And if I just, you know...	如果我只是,你知道的。 。 。
182	It could be individually less special, but there is so much more you can do.	它可能并不那么特别, 但你可以做的更多。
183	Right now, you can only make plans to do something for a decade or two.	现在,你只能做一个计划 做一个十两年。
184	You cannot have an ambitious plan of working on this project for 500 years.	在500年的时间里,你不可能有一个雄心勃勃的计划来实施这个项目。
185	Imagine possibilities open to you with infinite time in the infinite universe.	想象一下在无限宇宙中 无限时间给你带来的可能性
186	Gosh.	天呐
187	Well, you can.	嗯,你可以做到这一点。
188	It feels exhausting.	感觉很累
189	It's a big amount of time.	时彼大乘.
190	Also, I don't know about you, but I don't remember like 99% of my life in detail.	而且,我不知道关于你, 但我不记得有99%的 生活细节。
191	I remember big highlights.	我还记得一些大事
192	So even if I enjoyed Hawaii 10 years ago, I'll enjoy it again.	所以,即使我享受夏威夷 十年前,我会再次享受它。
193	你是否真的在想 和Brian Johnson一样
194	Brian Johnson is convinced that we're like maybe two decades away from being able to extend life.	布赖恩·约翰逊确信,我们可能还有20年的时间才能延长寿命.
195	Are you thinking about that practically?	你在考虑这个吗?
196	And are you doing anything about it?	你在做任何事情吗?
197	Diet, nutrition.	饮食,营养。
198	I try to think about investment strategies which pay out in a million years.	我试图思考投资策略 如何支付在一百万年。
199	Really?	真的吗?
200	Yeah, of course.	当然
201	What do you mean, of course?	你什么意思,当然?
202	Why wouldn't you?	为什么不呢?
203	If you think this is what's going to happen, you should try that.	如果你觉得事情会这样 你应该试试看
204	So if we get AI right, now what happens to economy?	所以,如果我们得到AI的权利, 现在怎么样的经济?
205	We talked about world coin.	我们谈的是世界硬币。
206	We talked about free labor.	我们讨论过自由劳动
207	What's money?	钱是什么?
208	Is it now Bitcoin?	是比特币吗?
209	Do you invest in that?	你投资吗?
210	Is there something else which becomes the only resource we cannot fake?	还有别的东西成为我们无法伪造的唯一资源吗?
211	So those things are very important research topics.	所以,这些东西是非常重要的研究课题。
212	So you're investing in Bitcoin, aren't you?	所以你在投资比特币 是吗?
213	Yeah.	对
214	Because it's the only scarce resource.	因为它是唯一稀缺的资源。
215	Nothing else has scarcity.	没有什么是缺少。
216	Everything else, if price goes up, will make more.	其他一切,如果价格上升,会增加。
217	I can make as much gold as you want given a proper price point.	我可以给你一个合适的价点 做多少黄金
218	You cannot make more Bitcoin.	你不能让更多的比特币。
219	Some people say Bitcoin is just this thing on a computer that we all agreed was valuable.	有人说比特币只是我们一致认为有价值的电脑上的东西。
220	We are a thing on a computer.	我们是电脑上的东西。
221	Remember?	记得吗?
222	Okay.	摆
223	So, I mean, not investment advice, but investment advice.	所以,我的意思是,不是投资建议,而是投资建议。
224	It's hilarious how that's one of those things where they tell you it's not, but you know it is immediately.	好笑的是,那是其中之一 他们告诉你它不是,但你知道这是立即。
225	Your call is important to us.	你的决定对我们很重要
226	That means your call is of zero importance and investment is like that.	这意味着你的呼吁是没有意义的 投资就是这样的
227	Yeah, when they say no investment advice, it's definitely investment advice, but it's not investment advice.	是啊,当他们说没有投资建议, 这绝对是投资建议, 但这不是投资建议。
228	Okay, so you'll bullish on Bitcoin because it can't be messed with.	好吧,所以你会欺负 比特币,因为它是不能搞砸了。
229	It is the only thing which we know how much there is in the universe.	这是我们唯一知道宇宙有多少的东西。
230	So gold, there could be an asteroid made out of pure gold heading towards us, devaluing it.	因此,金,可能有一个小行星 由纯金制成 朝我们的方向, 贬低它的价值。
231	Well, also killing all of us, but Bitcoin, I know exactly the numbers.	我也杀了我们所有人 但比特币 我知道确切的数字
232	And even the 21 million is an upper limit.	乃至二千一百万为上限.
233	How many are lost?	有多少人失踪了?
234	Passwords forgotten.	密码被遗忘了
235	I don't know what Satoshi is doing with his million.	我不知道佐藤用他那百万来做什么
236	It's getting scarcer every day while more and more people are trying to accumulate it.	越来越少了,而越来越多的人试图积累它。
237	Some people worry that it could be hacked with a supercomputer.	有些人担心它可能被超级计算机黑入.
238	A quantum computer can break that algorithm.	量子计算机可以打破算法.
239	There is strategies for switching to quantum resistant cryptography for that.	有策略可以切换到 量子抗性密码学。
240	And quantum computers are still kind of weak.	量子计算机仍然很薄弱。
241	Do you think there's any changes to my life that I should make following this conversation?	你觉得这次谈话之后 我的生活会有什么改变吗?
242	Is there anything that I should do differently the minute I walk out of this door?	当我走出这扇门时,我还能做点别的吗?
243	I assume you already invest in Bitcoin heavily.	我猜你已经对比特币投资很大了
244	Yes, I'm an investor in Bitcoin.	是的,我是一个投资 比特币。
245	This is financial advice.	这是财务建议。
246	No, just you seem to be winning.	不,只是你似乎赢了。
247	Maybe it's your simulation.	也许是你的模拟。
248	You're rich, handsome.	你有钱,帅哥
249	You have famous people hang out with you.	你们有名人陪你玩
250	Like that's pretty good.	感觉不错
251	Keep it up.	继续
252	Robin Hansen has a paper about how to live in a simulation, what you should be doing in it.	Robin Hansen有一篇关于模拟中如何生活的文章,你应该做些什么。
253	And your goal is to do exactly that.	你的目标是做到这一点。
254	You want to be interesting.	你想成为有趣的。
255	You want to hang out with famous people so they don't shut it down.	你想和名人在一起 这样他们就不会关闭它
256	So you are part of a part someone's actually watching in pay-per-view or something like that.	所以,你是一个角色的一部分 某人实际上在付费观看 或类似的东西。
257	Oh, I don't know if you want to be watched on pay-per-view because then...	哦,我不知道你是否想 在付费观赏中观看 因为...
258	Then they shut you down.	然后他们把你关了
259	If no one's watching, why would they play it?	如果没人在看,他们为什么要玩?
260	I'm saying, don't you want to fly under the radar?	我是说 你不想在雷达下飞行吗?
261	Don't you want to be the guy that's just living a normal life that the master...	难道你不想做一个 过着正常生活的人吗?
262	Those are NPCs.	这些是国家计委。
263	Nobody wants to be an NPC.	没有人想成为国家警察局。
264	Are you religious?	你信教吗?
265	Not in any traditional sense, but I believe in simulation hypothesis, which has a super intelligent being.	不是任何传统意义上的, 但我相信模拟假说, 它有一个超级智能的存在。
266	But you don't believe in the like, you know, the religious books.	但你不相信 类似,你知道,宗教书籍。
267	So different religions.	如此不同诸教.
268	This religion will tell you, don't work Saturday.	这个宗教会告诉你,不要工作周六。
269	This one, don't work Sunday.	这个星期天别工作
270	Don't eat pigs.	别吃猪
271	Don't eat cows.	别吃牛.
272	They just have local traditions on top of that theory.	他们只是有当地的传统 之上的理论。
273	That's all it is.	就是这样
274	They all the same religion.	二者皆同教义.
275	They all worship super intelligent being.	他们都崇拜超级智慧的人
276	They all think this world is not the main one.	他们都认为这个世界不是主要的世界。
277	And they argue about which animal not to eat.	他们争论哪些动物不吃。
278	Skip the local flavors, concentrate on what do all the religions have in common.	省略当地风味, 专心于所有宗教的共同之处。
279	And that's the interesting part.	这是有趣的部分。
280	They all think there is something greater than humans, very capable, all knowing, all powerful.	他们都认为有比人类更伟大的东西,非常有能力,所有人都知道,所有的力量.
281	Then I run a computer game for those characters in the game.	然后,我运行一个电脑游戏 在游戏中的角色。
282	I am that.	我是这样的。
283	I can change the whole world.	我可以改变整个世界
284	I can shut it down.	我可以关闭它。
285	I know everything in the world.	知一切世间.
286	It's funny.	这很有趣。
287	I was thinking earlier on when we started talking about the simulation theory that there's, there might be something innate in us that has been left from the creator, almost like a clue, like an intuition.	之前我在想,当我们开始谈论模拟理论的时候, 可能我们身上有某种内在的东西 从创造者身上留下的, 几乎像一个线索,像一个直觉。
288	Because that's what we tend to have through history.	因为我们在历史中往往如此。
289	Humans have this intuition.	人类有这种直觉。
290	That all the things you said are true.	你说的一切都是真的
291	That there's this somebody above and that.	上面还有一个人
292	We have generations of people who were religious, who believed God told them and was there and give them books.	我们有几代人信奉宗教,他们相信上帝告诉他们,他们在那里,给他们书。
293	And that has been passed on for many generations.	这已经传承了好几代人。
294	This is probably one of the earliest generations not to have universal religious belief.	这也许是没有普遍宗教信仰的最早的一代之一.
295	I wonder if those people are telling the truth.	我想知道那些人说的是不是真的
296	I wonder if those people that say God came to them and said something.	我想知道那些说上帝的人 会不会来找我说些什么
297	Imagine that.	试想一下
298	Imagine if that was part of this.	想象一下如果这是其中的一部分。
299	I'm looking at the news today.	我今观此新闻.
300	Something happened an hour ago.	一个小时前发生了什么
301	And I'm getting different conflicting results.	我得到了不同的冲突结果。
302	I can't even get with cameras, with drones, with like guy on Twitter there.	我连摄像头,无人机, 和推特上的人一样都买不到
303	I still don't know what happened.	我还是不知道发生了什么
304	And you think 3000 years ago we have accurate record of translations?	你觉得3000年前我们有准确的翻译记录吗?
305	No, of course not.	不,当然不是。
306	You know, these conversations you have around AI safety, do you think they make people feel good?	你知道,这些谈话 你围绕AI安全, 你觉得他们 让人感觉很好?
307	I don't know if they feel good or bad, but people find it interesting.	我不知道他们是好是坏 但人们觉得这很有趣
308	It's one of those topics.	这是其中的一个话题。
309	So I can have a conversation about different cures for cancer with an average person, but everyone has opinions about AI.	所以我可以和普通人谈谈不同的癌症治疗方法,但每个人都对AI有看法.
310	Everyone has opinions about simulation.	每个人都对模拟有看法。
311	It's interesting that you don't have to be highly educated or a genius to understand those concepts.	有趣的是,你不需要受过高学历或天才才能理解这些概念.
312	Because I tend to think that it makes me feel not positive.	因为我倾向于认为 这让我觉得不积极。
313	And I understand that, but I've always been of the opinion that you shouldn't live in a world of delusion where you're just	我理解,但我一直认为 你不应该生活在一个幻想的世界 你只是
314	seeking to be positive, have sort of positive things said and avoid uncomfortable conversations.	并避免不愉快的对话。
315	Actually, progress often in my life comes from like having uncomfortable conversations, becoming aware about something, and then at least being informed about how I can do something about it.	事实上,我生命中的进步 往往来自于一些不愉快的谈话, 开始意识到一些事情, 并且至少被告知我如何做一些事情。
316	And so I think that's why I asked the question, because I assume most people should, if they're normal human beings, listen to these conversations and go, gosh, that's scary.	所以我认为这就是我问这个问题的原因, 因为我认为大多数人,如果他们是正常的人, 应该听这些对话,然后去,天哪,那太可怕了。
317	And this is concerning.	这件事很重要
318	And then I keep coming back to this point, which is like, what do I do with that energy?	然后,我继续回到这个点, 也就是,我用这种能量做什么?
319	Yeah, but I'm trying to point out this is not different than so many conversations.	是的 但我想指出 这跟很多对话没什么不同
320	We can talk about, oh, there is starvation in this region, genocide in this region, you're dying, cancer is spreading, autism is up.	我们可以谈论,哦,这个地区有饥饿, 这个地区的种族灭绝, 你正在死亡,癌症在蔓延,自闭症在上升。
321	You can always find something to be very depressed about and nothing you can do about it.	你总能找到非常沮丧的东西 却无能为力
322	And we are very good at concentrating on what we can change, what we are good at, and basically not trying to embrace the whole world as a local environment.	我们非常善于专注于我们能改变的事物, 我们擅长的事物, 并且基本上不试图将整个世界作为一个地方环境。
323	So historically, you grew up with a tribe, you had a dozen people around you, if something happened to one of them, it was very rare, it was an accident.	所以,从历史上讲,你和一个部落一起长大, 你身边有十几个人, 如果其中一个人出了什么事, 这是非常罕见的,这是一个意外。
324	Now, if I go on the Internet, somebody gets killed everywhere all the time.	现在,如果我上互联网, 有人总是在任何地方被杀。
325	Somehow, thousands of people are reported to me every day.	不知何故,每天都有成千上万的人向我报告。
326	I don't even have time to notice.	我什至没有时间注意到。
327	It's just too much.	这太过分了
328	So I have to put filters in place.	所以我必须安装过滤器。
329	And I think this topic is what people are very good at filtering as like, this was this entertaining talk I went to, kind of like a show.	我认为这个话题是人们非常擅长过滤的, 这是我去过的娱乐性演讲, 有点像一场演出。
330	And the moment I exit, it ends.	我一离开就结束了
331	So usually I would go give a keynote at a conference and I tell them, basically, you're going to die, you have two years left, any questions?	所以通常我会在会议上做一个主旨发言,然后我告诉他们, 基本上,你会死,你还有两年时间,还有问题吗?
332	And people will be like, will I lose my job?	人们会说,我会失去我的工作吗?
333	How do I lubricate my sex robot?	我如何润滑我的性机器人?
334	Like all sorts of nonsense, clearly not understanding what I'm trying to say there.	像各种胡说八道,显然不理解我在说什么。
335	And those are good questions, interesting questions, but not fully embracing the result.	这些都是很好的问题,有趣的问题, 但并不完全接受结果。
336	They're still in their bubble of local versus global.	他们仍然在他们 的泡沫 地方与全球。
337	And the people that disagree with you the most as it relates to AI safety, what is it that they say?	那些最不认同你的人 他们说什么来着?
338	What are their counter arguments typically?	他们的辩驳通常是什么?
339	So many don't engage at all, like they have no background knowledge in a subject.	所以很多人根本不参与, 就像他们没有关于一个课题的背景知识。
340	They never write a single book, single paper, not just by me, by anyone.	他们从不写一本书,一张纸,不只是我写的,还有任何人写的。
341	They may be even working in a field.	他们甚至可能在一个领域工作。
342	So they are doing some machine learning work for some company, maximizing ad clicks.	因此,他们正在做一些机器学习工作 一些公司, 最大化的广告点击。
343	And to them, those systems are very narrow.	对他们来说,这些系统非常狭窄。
344	And then they hear that, obviously AI is going to take over the world.	然后他们听到,显然AI会接管世界。
345	They're like, it has no hands.	他们就像, 它没有手。
346	How would it do that?	怎么会这样?
347	It's nonsense.	胡说八道.
348	This guy is crazy, he has a beard.	这家伙疯了,他有胡子。
349	Why would I listen to him?	我为什么要听他的?
350	That's, then they start reading a little bit.	也就是说,他们开始读一些东西。
351	They go, oh, okay.	他们走了,哦,好吧。
352	So maybe AI can be dangerous.	也许AI是危险的
353	Yeah, I see that.	是的,我看见了。
354	But we always solve problems in the past.	但我们过去总是解决问题。
355	We're going to solve them again.	我们要再次解决他们。
356	I mean, at some point we fixed a computer virus or something.	我的意思是,在某个时候,我们固定 计算机病毒什么的。
357	So it's the same.	故曰无异.
358	And basically the more exposure they have, the less likely they are to keep that position.	基本上,他们接触的越多, 保持这种位置的可能性就越小。
359	I know many people who went from super careless developer to safety researcher.	我认识很多从超级粗心的开发者到安全研究员的人.
360	I don't know anyone who went from, I worry about AI safety to like, there is	我不知道谁从, 我担心AI的安全 喜欢,有
361	nothing to worry about.	没什么好担心的
362	What are your closing statements?	你的闭幕词是什么?
363	Let's make sure there is not a closing statement we need to give for humanity.	让我们确保我们不需要为人类发表结束语。
364	Let's make sure we stay in charge, in control.	让我们确保我们负责,控制。
365	Let's make sure we only build things which are beneficial to us.	让我们确保我们只建造 对我们有好处的东西。
366	Let's make sure people who are making those decisions are remotely qualified to do it.	让我们确保那些做出这些决定的人 完全有资格这样做。
367	They are good, not just at science, engineering and business, but also	不仅在科学、工程和企业方面都是好的,而且
368	have moral and ethical standards.	拥有道德和道德标准。
369	And if you're doing something which impacts other people, you should ask	如果你正在做一些影响别人的事 你应该问
370	their permission before you do that.	在你这样做之前,他们的许可。
371	If there was one button in front of you, and it would shut down every AI company in	如果有一个按钮在你面前, 它会关闭每一个AI公司在
372	the world right now, permanently, with the inability for anybody to start a new one,	现在的世界,永久, 任何人都无法开始一个新的,
373	would you press the button?	你能按下按钮吗?
374	Are we losing narrow AI or just super intelligent AGI part?	我们是失去狭义的AI 还是只是超智能的AGI部分?
375	Losing all of AI.	失去所有人工智能。
376	That's a hard question because AI isn't extremely important.	这是一个很难的问题,因为AI不是极其重要的.
377	It controls stock market power plants.	它控制股票市场发电厂.
378	It controls hospitals.	它控制着医院
379	It would be a devastating accident.	这将是一个毁灭性的事故。
380	Millions of people would lose their lives.	数百万人将丧生。
381	Okay.	摆
382	We can keep narrow AI.	我们可以保持狭窄的AI。
383	Oh yeah.	对
384	That's what we want.	这是我们想要的。
385	We want narrow AI to do all this for us, but not God we don't	我们希望狭义的AI为我们做这一切, 但上帝我们不
386	control doing things to us.	控制我们的行为。
387	So you would stop it, you would stop AGI and super intelligence.	所以你会阻止它, 你会阻止AGI和超级智能。
388	我们有AGI。
389	What we have today is great for almost everything.	我们今天拥有的几乎是一切。
390	We can make secretaries out of it.	我们可以做一些秘书
391	99% of economic potential of current technology has not been deployed.	目前技术的经济潜力有99%没有利用。
392	We make AI so quickly, it doesn't have time to propagate through the industry,	我们做AI这么快, 它没有时间通过行业传播,
393	through technology, something like half of all jobs are considered BS jobs.	通过技术,大约一半的工作 被认为是BS的工作。
394	They don't need to be done, bullshit jobs.	他们不需要做,狗屎工作。
395	So those can be not even automated.	因此,这些甚至不能自动化。
396	They can be just gone.	他们可能只是走了。
397	But I'm saying we can replace 60% of jobs today with existing models.	但我想说,今天我们可以用现有的模型来取代60%的工作.
398	We're not done that.	我们没有做。
399	So if the goal is to grow economy, to develop, we can do it for decades	因此,如果目标是 增长经济,发展, 我们可以做到这一点几十年
400	without having to create super intelligence as soon as possible.	不需要尽快创造出超级智能
401	Do you think globally, especially in the Western world, unemployment's	你认为全球,特别是在西方世界, 失业是
402	only going to go up from here?	只从这里上去?
403	Do you think relatively this is the low of unemployment?	你认为这是相对的低失业率吗?
404	I mean, it fluctuates a lot with other factors.	我的意思是,它波动很多 其他因素。
405	There are wars, there is economic cycles, but overall, the more jobs you automate	有战争,有经济周期, 但总的来说,你自动化的工作越多
406	and the higher is the intellectual necessity to start a job, the fewer people qualify.	而创业的智力需求越高,资格越低。
407	So if we plotted it on a graph over the next 20 years, you're assuming	所以,如果我们在图上绘制 它在未来20年,你假设
408	unemployment's gradually going to go up over that time?	失业率会逐渐上升吗?
409	I think so.	我想是这样。
410	Fewer and fewer people would be able to contribute.	能够作出贡献的人越来越少。
411	Already, we kind of understand it because we created minimum wage.	我们已经理解了,因为我们创造了最低工资。
412	We understood some people don't contribute enough economic value	我们知道有些人没有贡献足够的经济价值
413	to get paid anything really.	以获得任何真正的报酬。
414	So we had to force employers to pay them more than they're worth.	所以我们不得不强迫雇主 支付他们比他们值钱。
415	And we haven't updated it.	我们还没更新呢
416	It's what, $7.25 federally in the US.	这是什么,7.25联邦在美国。
417	If you keep up with the economy, it should be like $25 an hour now, which	如果你能跟上经济, 应该是像每小时25美元,
418	means all these people making less are not contributing enough economic	意味着所有那些做更少的人 都没有足够的经济贡献
419	output to justify what they're getting paid.	以证明他们得到的报酬
420	We have a closing tradition on this podcast where the last guest leaves	我们有一个关闭的传统 在这个播客 最后的客人离开
421	a question for the next guest, not knowing who they're leaving it for.	给下一位客人的问话 不知道他们要留给谁
422	And the question left for you is what are, what are the most important	剩下的问题是 最重要的是什么
423	characteristics for a friend, colleague, or mate?	朋友、同事还是朋友的特征?
424	Those are very different types of people, but for all of them,	这几类人非常不同 但对他们所有人来说
425	loyalty is number one.	忠诚是第一
426	And what does loyalty mean to you?	忠诚对你意味着什么?
427	Not betraying you, not screwing you, not cheating on you.	不背叛你,不和你上床,不欺骗你。
428	Despite the temptation.	虽有诱惑.
429	Despite the world being as it is, situation, environment.	尽管世界是一样, 情况,环境。
430	Dr. Roman, thank you so much.	罗曼医生,非常感谢
431	Thank you so much for doing what you do because you're, you're starting	非常感谢你所做的一切 因为你,你开始
432	a conversation and pushing forward a conversation and doing research	对话 推进对话 进行研究
433	that is incredibly important.	这是非常重要的。
434	And you're doing it in the face of a lot of, a lot of skeptics, I'd say.	且汝当以多有,多有疑论者,为所为.
435	There's a lot of people that have a lot of incentives to discredit what	有很多人有很强的动机去诋毁什么
436	you're saying and what you do because they have their own incentives and	因为他们有自己的动机
437	they have billions of dollars on the line and they have their jobs on	他们有数十亿美元 在线和他们的工作
438	the line potentially as well.	线也有可能。
439	So it's really important that there are people out there that are willing	因此,真正重要的是,有 人在那里愿意
440	to, I guess, stick their head above the parapet and come on shows like this	把他们的头贴在栏杆上 像这样来表演
441	and go on big platforms and talk about the unexplainable, unpredictable,	在大平台上谈论那些无法解释 无法预测的事
442	uncontrollable future that we're heading towards.	我们即将走向的不可控制的未来。
443	So thank you for doing that.	所以,谢谢你这样做。
444	This book, which, which I think everybody should check out if they want	这本书,我认为每个人都应该 检查他们是否想要
445	a continuation of this conversation, I think was published in 2024, gives	我认为是2024年出版的,
446	a holistic view on many of the things we've talked about today, preventing	对于我们今天谈论的许多事情, 一个整体的观点, 防止
447	AI failures and much, much more.	AI失败,还有更多
448	And I'm going to link it below for anybody that wants to read it.	我会把它连到下面 给任何想读它的人
449	If people want to learn more from you, if they want to go further into your	如果人们想从你身上学到更多的东西, 如果他们想深入到你
450	work, what's the best thing for them to do?	工作,什么是他们最好做的吗?
451	Where do they go?	他们去哪里?
452	They can follow me, follow me on Facebook, follow me on X, just don't	他们可以跟着我,在Facebook上跟着我,在X上跟着我,只是不要
453	follow me home, very important.	跟着我回家,非常重要
454	Okay.	摆
455	So I'll put your Twitter, your X account as well below so people can follow you	所以,我会把你的 推特,你的X账户 以及下面的人可以跟踪你
456	there and yeah, thank you so much for doing what you do, it's remarkably	在那里,是的,非常感谢 做你的工作,这是显著的
457	eye-opening and it's given me so much food for thought and it's actually	打开眼睛,它给了我这么多食物 思考,实际上
458	convinced me more that we are living in a simulation, but it's also made me	更让我相信 我们生活在模拟中,但它也让我
459	think quite differently of religion, I have to say, because you're right, all	我不得不说,你对宗教的看法很不同,因为你是对的
460	the religions, when you get away from the sort of the local traditions, they	宗教,当你远离 那种地方传统,他们
461	do all point at the same thing and actually if they are all pointing at the	做所有点 在相同的事情,实际上 如果他们都指
462	same thing, then maybe the fundamental truths that exist across them should be	同样的,那么也许 基本真理 存在于他们身上
463	something I pay more attention to, things like loving thy neighbor, things	我更关心一些事情 比如爱你的邻居
464	like the fact that we are all one, that there's a divine creator and maybe	就像我们都是一体一样 有一个神的创造者 也许
465	also they all seem to have consequence beyond this life, so maybe I should be	他们似乎都有 后果过后的生活, 所以也许我应该
466	thinking more about how I behave in this life and where I might end up there	更想想我这一辈子的作风 以及我最后会到哪里去
467	after.	后结.
468	Roman, thank you.	罗曼,谢谢
469	Amen.	阿门

该视频共有字幕 469 条。解锁更多字幕为会员功能，请移动到 价格