imported>Willy1018-bot：BOTR：批量替換所有lang-xx模板為langx模板

2025-06-25T12:20:39Z

BOTR：批量替換所有lang-xx模板為langx模板

新页面

{{noteTA|G1=IT}}
{{Infobox software
| title = 生成型预训练变换模型 3<br />Generative Pre-trained Transformer 3（GPT-3）
| name =
| logo =
| author = [[OpenAI]]
| released = {{Start date and age|2020|06|11}} (beta)
| latest release = {{Start date and age|2023|02|13}}
| latest release version =
| latest release date =
| repo = https://github.com/openai/gpt-3
| programming language =
| operating system =
| replaces = [[GPT-2]]
| replaced_by = [[GPT-4]]
| language count =
| genre ={{indented plainlist |
*[[大型语言模型]]
*[[基于转换器的生成式预训练模型]]
*[[基礎模型]]
}}
| license = [[专有软件|专有]]
| website = {{URL|https://openai.com/blog/openai-api}}
| AsOf =
}}

'''生成型预训练變換模型 3''' （{{langx|en|Generative Pre-trained Transformer 3}}，簡稱 '''GPT-3'''）是一個[[自回歸模型|自迴歸]][[語言模型]]，目的是為了使用[[深度學習]]生成人類可以理解的自然語言<ref name="arXiv_Brown_20200722">{{Cite journal |last=Brown |first=Tom B. |last2=Mann |first2=Benjamin |last3=Ryder |first3=Nick |last4=Subbiah |first4=Melanie |last5=Kaplan |first5=Jared |last6=Dhariwal |first6=Prafulla |last7=Neelakantan |first7=Arvind |last8=Shyam |first8=Pranav |last9=Sastry |first9=Girish |last10=Askell |first10=Amanda |last11=Agarwal |first11=Sandhini |title=Language models are few-shot learners |url=https://dl.acm.org/doi/abs/10.5555/3495724.3495883 |journal=Proceedings of the 34th International Conference on Neural Information Processing Systems |series=NIPS '20 |location=Red Hook, NY, USA |publisher=Curran Associates Inc. |date=2020-12-06 |doi=10.5555/3495724.3495883 |isbn=978-1-7138-2954-6 |access-date=2025-01-14 |archive-date=2025-01-11 |archive-url=https://web.archive.org/web/20250111174154/https://dl.acm.org/doi/abs/10.5555/3495724.3495883 |dead-url=no }}</ref>。GPT-3是由在[[舊金山]]的[[人工智能]]公司[[OpenAI]]訓練與開發，模型設計基於[[谷歌]]開發的 [[變換語言模型|Transformer 語言模型]]。GPT-3的[[人工神经网络|神經網路]]包含1750億個參數，需要700GB来存储<ref name="ZDNet_Tiernan_20200601">{{Cite web| last = Ray| first = Tiernan| date = 2020-06-01| title = OpenAI’s gigantic GPT-3 hints at the limits of language models for AI| work = ZDNet| access-date = 2020-07-31| url = https://www.zdnet.com/article/openais-gigantic-gpt-3-hints-at-the-limits-of-language-models-for-ai/| archive-date = 2020-06-01| archive-url = https://web.archive.org/web/20200601081629/https://www.zdnet.com/article/openais-gigantic-gpt-3-hints-at-the-limits-of-language-models-for-ai/| dead-url = no}}</ref>。该模型在许多任务上展示了强大的[[零样本学习|零样本]]和[[提示工程|少样本]]的能力<ref name="OpenAI_Radford_20200611">{{Cite journal |last=Radford |first=Alec |last2=Narasimhan |first2=Karthik |title=Improving Language Understanding by Generative Pre-Training |url=https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035 |date=2018 |access-date=2025-01-14 |archive-date=2025-01-07 |archive-url=https://web.archive.org/web/20250107145045/https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035 |dead-url=no }}</ref>。

OpenAI於2020年5月發表GPT-3的論文，在次月為少量公司與開發人團釋出[[應用程式介面]]的測試版。[[微軟]]在2020年9月22日宣布取得了GPT-3的獨家授權<ref>{{Cite magazine |title=OpenAI is giving Microsoft exclusive access to its GPT-3 language model |url=https://www.technologyreview.com/2020/09/23/1008729/openai-is-giving-microsoft-exclusive-access-to-its-gpt-3-language-model/ |date=2020-09-23 |last=Hao |first=Karen |access-date=2020-09-25 |work=[[MIT Technology Review]] |language=en |quote="The companies say OpenAI will continue to offer its public-facing API, which allows chosen users to send text to GPT-3 or OpenAI’s other models and receive its output. Only Microsoft, however, will have access to GPT-3’s underlying code, allowing it to embed, repurpose, and modify the model as it pleases." |archive-date=2021-02-05 |archive-url=https://web.archive.org/web/20210205121656/https://www.technologyreview.com/2020/09/23/1008729/openai-is-giving-microsoft-exclusive-access-to-its-gpt-3-language-model/ }}</ref>。

GPT-3被認為可寫出人類無法與電腦區別的文章與字串，GPT-3原始論文的作者们警告了GPT-3有可能對於社會的負面影響，比如利用製造[[假新聞]]的可能性。英國《[[衛報]]》即使用GPT-3生成了一個關於人工智慧对人类无威胁的評論專欄<ref>{{Cite news|last=GPT-3|date=2020-09-08|title=A robot wrote this entire article. Are you scared yet, human?|language=en-GB|work=The Guardian|url=https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3|access-date=2020-09-15|issn=0261-3077|archive-date=2021-02-04|archive-url=https://web.archive.org/web/20210204102121/https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3|dead-url=no}}</ref>。[[李開復]]稱[[卷積神經網路]]與GPT-3為人工智能重要的改善，两者皆是模型加海量数据的成果<ref>{{Cite web|last=Lee|first=Kai-fu|url=https://www.linkedin.com/pulse/%E6%9D%8E%E5%BC%80%E5%A4%8D%E5%AF%B9%E8%AF%9Dmit%E5%BD%AD%E7%89%B9%E5%85%B0ai%E4%B8%8D%E6%98%AF%E5%8D%95%E6%89%93%E7%8B%AC%E6%96%97%E5%BA%94%E9%81%BF%E5%85%8Dai%E5%86%B7%E6%88%98-kai-fu-lee/|date=2020-10-09|access-date=2020-10-17|title=李开复对话MIT彭特兰：AI不是单打独斗，应避免AI冷战|archive-date=2024-10-04|archive-url=https://web.archive.org/web/20241004112152/https://www.linkedin.com/pulse/%E6%9D%8E%E5%BC%80%E5%A4%8D%E5%AF%B9%E8%AF%9Dmit%E5%BD%AD%E7%89%B9%E5%85%B0ai%E4%B8%8D%E6%98%AF%E5%8D%95%E6%89%93%E7%8B%AC%E6%96%97%E5%BA%94%E9%81%BF%E5%85%8Dai%E5%86%B7%E6%88%98-kai-fu-lee/|dead-url=no}}</ref>。

==背景==

据《[[经济学人]]》报道，改进的算法、强大的计算机和数字化数据的增加推动了机器学习的革命，2010 年代的新技术导致“任务的快速改进”，包括操纵语言。<ref name="theeconomist_20200611">{{Cite news| issn = 0013-0613| title = An understanding of AI's limitations is starting to sink in| newspaper = The Economist| date = 2020-06-11| access-date = 2020-07-31| url = https://www.economist.com/technology-quarterly/2020/06/11/an-understanding-of-ais-limitations-is-starting-to-sink-in| archive-date = 2020-07-31| archive-url = https://web.archive.org/web/20200731060114/https://www.economist.com/technology-quarterly/2020/06/11/an-understanding-of-ais-limitations-is-starting-to-sink-in| dead-url = no}}</ref>

==训练和能力==

在2020年5月28日，由OpenAI团队中31名工程师和研究人员撰写的arXiv预印本介绍了第三代“最先进的语言模型”——GPT-3的开发情况。<ref name="analyticsindiamag_Sagar_20200603">{{Cite magazine| last = Sagar| first = Ram| title = OpenAI Releases GPT-3, The Largest Model So Far| magazine = Analytics India Magazine| access-date = 2020-07-31| date = 2020-06-03| url = https://analyticsindiamag.com/open-ai-gpt-3-language-model/| archive-date = 2020-08-04| archive-url = https://web.archive.org/web/20200804173452/https://analyticsindiamag.com/open-ai-gpt-3-language-model/| dead-url = no}}</ref>该团队相对于GPT-2，<ref name="gpt2-with-quote">{{cite web |title=Language Models are Unsupervised Multitask Learners |url=https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf |access-date=2019-12-04 |quote="GPT-2, is a 1.5B parameter Transformer" |website=openai.com |archive-date=2019-12-12 |archive-url=https://web.archive.org/web/20191212223916/https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf |dead-url=no }}</ref>将GPT-3的容量增加了两个数量级以上，使其成为迄今为止最大的非稀疏语言模型。<ref name="neurips_Brown_202012">{{cite journal|first1=Tom B.|last1=Brown|first2=Benjamin|last2=Mann|last3=Ryder|last25=Chess|last20=Hesse|first20=Christopher|last21=Chen|first21=Mark|last22=Sigler|first22=Eric|last23=Litwin|first23=Mateusz|last24=Gray|first24=Scott|first26=Jack|first25=Benjamin|last26=Clark|last19=Winter|last27=Berner|first27=Christopher|last28=McCandlish|first28=Sam|last29=Radford|first29=Alec|last30=Sutskever|first30=Ilya|last31=Amodei|first31=Dario|first19=Clemens|first18=Jeffrey|first3=Nick|last10=Askell|last4=Subbiah|first4=Melanie|last5=Kaplan|first5=Jared|last6=Dhariwal|first6=Prafulla|last7=Neelakantan|first7=Arvind|last8=Shyam|first8=Pranav|last9=Sastry|first9=Girish|first10=Amanda|last18=Wu|last11=Agarwal|first11=Sandhini|last12=Herbert-Voss|first12=Ariel|last13=Krueger|first13=Gretchen|last14=Henighan|first14=Tom|last15=Child|first15=Rewon|last16=Ramesh|first16=Aditya|last17=Ziegler|first17=Daniel M.|editor1-last=Larochelle|editor1-first=H.|editor2-last=Ranzato|editor2-first=M.|editor3-last=Hadsell|editor3-first=R.|editor4-last=Balcan|editor4-first=M.F.|editor5-last=Lin|editor5-first=H.|title=Language Models are Few-Shot Learners|journal=Advances in Neural Information Processing Systems|date=Dec 2020|volume=33|pages=1877–1901|url=https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf|publisher=Curran Associates, Inc.|access-date=2023-02-16|archive-date=2023-05-25|archive-url=https://web.archive.org/web/20230525115303/https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf|dead-url=no}}</ref>{{rp|14|quote="Since we increase the capacity by over two orders of magnitude from GPT-2 to GPT-3"}}<ref name="CNBC_Shead_20200723">{{Cite news| last = Shead| first = Sam| title = Why everyone is talking about the A.I. text generator released by an Elon Musk-backed lab| work = CNBC| access-date = 2020-07-31| date = 2020-07-23| url = https://www.cnbc.com/2020/07/23/openai-gpt3-explainer.html| archive-date = 2020-07-30| archive-url = https://web.archive.org/web/20200730123130/https://www.cnbc.com/2020/07/23/openai-gpt3-explainer.html| dead-url = no}} Four preprints were released between May 28 and July 22, 2020.</ref>

斯坦福大学的研究发现，GPT3已经可以解决70%的[[心智理论]]任务，相当于7岁儿童；至于GPT3.5（ChatGPT的同源模型），更是解决了93%的任务，心智相当于9岁儿童。但这并不意味着，ChatGPT就真正具备了[[心智理论]]。可能它即使不被设计到AI系统中，也可以作为“副产品”通过训练得到。因此，相比探究GPT-3.5是不是真的有了心智还是像有心智，更需要反思的是这些测试本身。<ref name="Theory of Mind">{{cite web |title=Theory of Mind May Have Spontaneously Emerged in Large Language Models, Michal Kosinski |url=https://arxiv.org/abs/2302.02083 |access-date=2023-02-16 |archive-date=2023-03-06 |archive-url=https://web.archive.org/web/20230306134520/https://arxiv.org/abs/2302.02083 |dead-url=no }}</ref><ref>{{cite web |title=ChatGPT背后模型被证实具有人类心智，学术圈轰动 |url=https://www.fortunechina.com/jingxuan/26780.htm |access-date=2023-02-16 |archive-date=2023-03-06 |archive-url=https://web.archive.org/web/20230306223122/https://www.fortunechina.com/jingxuan/26780.htm |dead-url=no }}</ref>

== 迭代版本 ==
=== GPT-3.5 ===
2022年3月1日，OpenAI在其API中提供了新版本的GPT-3和[[OpenAI Codex|Codex]]，名称为“text-davinci-003”和“code-davinci-002”，具有编辑和插入的功能。<ref>{{Cite web |date=2022-03-15 |title=New GPT-3 Capabilities: Edit & Insert |url=https://openai.com/blog/gpt-3-edit-insert/ |access-date=2023-01-13 |website=OpenAI |language=en |archive-date=2023-01-13 |archive-url=https://web.archive.org/web/20230113234402/https://openai.com/blog/gpt-3-edit-insert/ |dead-url=no }}</ref>
=== GPT-3.5 Turbo ===
{{Empty section}}

== 参见 ==
{{div col|4}}
* [[OpenAI]]
* [[ChatGPT]]
* [[提示工程]]
* [[悟道预训练模型]]
* [[聊天机器人]]
* [[通用人工智能]]
* [[图灵测试]]
* [[生成对抗网络]]
* [[Transformer模型]]
* [[注意力机制]]
* [[對話程式語言模型|LaMDA(對話語言模型)]]
* [[深度学习]]
* [[OpenAI Codex]]
* [[GitHub Copilot]]
{{div col end}}

== 參考資料 ==
{{Reflist|2}}

== 外部链接 ==
* {{Official website}}
* [https://platform.openai.com/playground Playground for GPT3.5 and Codex] {{Wayback|url=https://platform.openai.com/playground |date=20230610092836 }}

{{OpenAI}}
{{Differentiable computing}}

[[Category:OpenAI]]
[[Category:交互式叙事]]
[[Category:聊天机器人]]
[[Category:人工智能應用]]
[[Category:2022年面世的產品]]
[[Category:人工智慧]]
[[Category:機器學習]]
[[Category:大型语言模型]]
[[Category:基于转换器的生成式预训练模型]]

GPT-3 - 版本历史

imported>Willy1018-bot：​BOTR：批量替換所有lang-xx模板為langx模板

imported>Willy1018-bot：BOTR：批量替換所有lang-xx模板為langx模板