{"id":4121,"date":"2024-07-17T17:17:58","date_gmt":"2024-07-17T09:17:58","guid":{"rendered":"https:\/\/nullthought.net\/?p=4121"},"modified":"2025-03-03T12:25:32","modified_gmt":"2025-03-03T04:25:32","slug":"miles-cranmer%e7%9a%84%e6%bc%94%e8%ae%b2%ef%bc%9a%e7%94%a8%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e5%8f%91%e7%8e%b0%e7%a7%91%e5%ad%a6%e7%9c%9f%e7%9f%a5","status":"publish","type":"post","link":"https:\/\/nullthought.net\/?p=4121","title":{"rendered":"Miles Cranmer\u7684\u6f14\u8bb2\uff1a\u7528\u795e\u7ecf\u7f51\u7edc\u53d1\u73b0\u79d1\u5b66\u771f\u77e5"},"content":{"rendered":"\n<p><a href=\"https:\/\/x.com\/MilesCranmer\" target=\"_blank\" rel=\"noreferrer noopener\">Miles Cranmer<\/a>\u662f\u5251\u6865\u5927\u5b66\u52a9\u7406\u6559\u6388\uff0c\u4ed6\u4e8e2024\u5e744\u6708\u5728<strong><a href=\"https:\/\/www.simonsfoundation.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Simons Foundation<\/a><\/strong>\u53d1\u8868\u7684\u6f14\u8bb2<strong><a href=\"https:\/\/www.youtube.com\/watch?v=fk2r8y5TfNY\" target=\"_blank\" rel=\"noreferrer noopener\">The Next Great Scientific Theory is Hiding Inside a Neural Network<\/a><\/strong>\u5f88\u6709\u542f\u53d1\u6027\u3002<\/p>\n\n\n\n<p>\u4ffa\u5bf9\u89c6\u9891\u5185\u5bb9\u7684\u603b\u7ed3\uff1a\u5148\u201c\u5f62\u800c\u4e0b\u201d\uff0c\u518d\u201c\u5f62\u800c\u4e0a\u201d\u3002\u5148\u628a\u201c\u6a21\u5f0f\u201d\u538b\u7f29\u8fdbAI\u6a21\u578b\uff0c\u518d\u7528\u201c\u7b26\u53f7\u84b8\u998f\u201d\u63a2\u7a76\u201c\u6a21\u5f0f\u201d\u540e\u9762\u7684\u7406\u8bba\uff1a1\uff09\u5982\u4e3a\u5df2\u77e5\u7406\u8bba\u27a1\ufe0f\u53ef\u89e3\u91caAI\uff08XAI\uff09\uff1b2\uff09\u5982\u4e3a\u672a\u77e5\u89c4\u5f8b\u27a1\ufe0f\u53ef\u80fd\u662f\u7406\u8bba\u65b0\u53d1\u73b0\u3002<br>Summary for the video: first, compress the &#8220;patterns&#8221; into the AI model, and then use &#8220;symbolic distillation&#8221; to explore the theory behind the &#8220;patterns&#8221;: 1) If the theory is known \u27a1\ufe0f explainable AI (XAI); 2) If the theory is unknown \u27a1\ufe0f it may be a new theoretical discovery.<\/p>\n\n\n\n<figure class=\"wp-block-embed aligncenter is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Miles Cranmer - The Next Great Scientific Theory is Hiding Inside a Neural Network (April 3, 2024)\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/fk2r8y5TfNY?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption class=\"wp-element-caption\"><strong><a href=\"https:\/\/www.youtube.com\/watch?v=fk2r8y5TfNY\" target=\"_blank\" rel=\"noreferrer noopener\">The Next Great Scientific Theory is Hiding Inside a Neural Network<\/a><\/strong>, <a href=\"https:\/\/x.com\/MilesCranmer\" target=\"_blank\" rel=\"noreferrer noopener\">Miles Cranmer<\/a><\/figcaption><\/figure>\n\n\n\n<h5 class=\"wp-block-heading\"><em>Google Gemini\u5bf9\u89c6\u9891\u7684\u603b\u7ed3\uff1a<\/em><\/h5>\n\n\n\n<p>Miles Cranmer \u5728\u6f14\u8bb2\u4e2d\u63d0\u51fa\uff0c\u89e3\u91ca\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u6210\u4e3a\u53d1\u73b0\u79d1\u5b66\u65b0\u77e5\u7684\u65b0\u65b9\u6cd5\u3002 \u4ed6\u63d0\u51fa\u4e86\u4e00\u79cd\u540d\u4e3a\u7b26\u53f7\u84b8\u998f\u7684\u65b9\u6cd5\u6765\u5b9e\u73b0\u8fd9\u4e00\u76ee\u6807\u3002<\/p>\n\n\n\n<p>\u4f20\u7edf\u7684\u79d1\u5b66\u65b9\u6cd5\u662f\u5efa\u7acb\u7406\u8bba\u6765\u63cf\u8ff0\u6570\u636e\u3002 Cranmer \u8ba4\u4e3a\uff0c\u968f\u7740\u5f3a\u5927\u795e\u7ecf\u7f51\u7edc\u7684\u5174\u8d77\uff0c\u4e00\u79cd\u65b0\u7684\u65b9\u6cd5\u6210\u4e3a\u53ef\u80fd\u3002 \u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u8bad\u7ec3\u5927\u91cf\u6570\u636e\uff0c\u5e76\u53d1\u73b0\u73b0\u6709\u7406\u8bba\u4e2d\u672a\u5305\u542b\u7684\u6a21\u5f0f\u3002 \u6311\u6218\u5728\u4e8e\u4ece\u795e\u7ecf\u7f51\u7edc\u4e2d\u63d0\u53d6\u8fd9\u4e9b\u89c1\u89e3\u5e76\u5c06\u5b83\u4eec\u7eb3\u5165\u79d1\u5b66\u7406\u89e3\u3002<\/p>\n\n\n\n<p>Cranmer \u63d0\u51fa\u7b26\u53f7\u84b8\u998f\u4f5c\u4e3a\u5b9e\u73b0\u8fd9\u4e00\u76ee\u6807\u7684\u65b9\u6cd5\u3002 \u8be5\u65b9\u6cd5\u6d89\u53ca\u901a\u8fc7\u627e\u5230\u4e00\u7ec4\u590d\u5236\u7f51\u7edc\u884c\u4e3a\u7684\u65b9\u7a0b\u6765\u89e3\u91ca\u795e\u7ecf\u7f51\u7edc\u3002 \u7136\u540e\u53ef\u4ee5\u5206\u6790\u8fd9\u4e9b\u65b9\u7a0b\u6765\u7406\u89e3\u7f51\u7edc\u5b66\u4e60\u7684\u5e95\u5c42\u79d1\u5b66\u539f\u7406\u3002<\/p>\n\n\n\n<p>Cranmer \u627f\u8ba4\u7b26\u53f7\u84b8\u998f\u5b58\u5728\u5c40\u9650\u6027\uff0c\u7279\u522b\u662f\u5b83\u65e0\u6cd5\u627e\u5230\u590d\u6742\u7684\u7b26\u53f7\u6a21\u578b\u3002 \u4f46\u662f\uff0c\u4ed6\u8ba4\u4e3a\u8fd9\u662f\u79d1\u5b66\u53d1\u73b0\u7684\u4e00\u4e2a\u4ee4\u4eba\u5174\u594b\u7684\u65b9\u5411\uff0c\u5c24\u5176\u662f\u968f\u7740\u50cf <strong><a href=\"https:\/\/polymathic-ai.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Polymathic AI<\/a><\/strong> \u8fd9\u6837\u7684\u57fa\u7840\u6a21\u578b\u7684\u53d1\u5c55\u3002 \u8fd9\u4e9b\u6a21\u578b\u5728\u591a\u4e2a\u5b66\u79d1\u4e0a\u8bad\u7ec3\u4e86\u5927\u91cf\u6570\u636e\u96c6\uff0c\u5e76\u4e14\u53ef\u80fd\u5305\u542b\u5e7f\u6cdb\u5e94\u7528\u7684\u79d1\u5b66\u6a21\u578b\u3002<\/p>\n\n\n\n<p>\u6f14\u8bb2\u4ee5\u5bf9\u5f00\u653e\u95ee\u9898\u7684\u8ba8\u8bba\u7ed3\u675f\u3002 \u4e00\u4e2a\u95ee\u9898\u662f\u5982\u4f55\u5c06\u66f4\u4e00\u822c\u7684\u7b97\u6cd5\uff08\u8d85\u51fa\u7b80\u5355\u7684\u6570\u5b66\u65b9\u7a0b\uff09\u7eb3\u5165\u7b26\u53f7\u56de\u5f52\u3002 \u53e6\u4e00\u4e2a\u95ee\u9898\u662f\uff0c\u9884\u5148\u8bad\u7ec3\u795e\u7ecf\u7f51\u7edc\u5904\u7406\u67d0\u4e9b\u7c7b\u578b\u7684\u6570\u636e\u662f\u5426\u4f1a\u5bf9\u79d1\u5b66\u53d1\u73b0\u9020\u6210\u4e0d\u5229\u5f71\u54cd\u3002 Cranmer \u627f\u8ba4\u53ef\u80fd\u5b58\u5728\u5bf9\u6297\u6027\u793a\u4f8b\uff0c\u5176\u4e2d\u9884\u5148\u8bad\u7ec3\u67d0\u4e9b\u6570\u636e\u96c6\u53ef\u80fd\u4f1a\u4f7f\u53d1\u73b0\u79d1\u5b66\u89c1\u89e3\u66f4\u52a0\u56f0\u96be\uff0c\u4f46\u8fc4\u4eca\u4e3a\u6b62\u5c1a\u672a\u53d1\u73b0\u6b64\u7c7b\u793a\u4f8b\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"476\" src=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/07\/image-11-1024x476.png\" alt=\"\" class=\"wp-image-4122\" srcset=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/07\/image-11-1024x476.png 1024w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/07\/image-11-300x140.png 300w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/07\/image-11-768x357.png 768w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/07\/image-11.png 1492w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><strong><a href=\"https:\/\/www.youtube.com\/watch?v=fk2r8y5TfNY\" target=\"_blank\" rel=\"noreferrer noopener\">The Next Great Scientific Theory is Hiding Inside a Neural Network<\/a><\/strong>, <a href=\"https:\/\/x.com\/MilesCranmer\" target=\"_blank\" rel=\"noreferrer noopener\">Miles Cranmer<\/a><\/figcaption><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h5 class=\"wp-block-heading\"><em>Video summary by Google Gemini:<\/em><\/h5>\n\n\n\n<p>This talk by Miles Cranmer argues that interpreting neural networks can be a new way to discover scientific insights. Cranmer proposes a method called symbolic distillation to achieve this.<\/p>\n\n\n\n<p>The traditional scientific approach involves building theories to describe data. Cranmer argues that with the rise of powerful neural networks, a new approach is possible. Neural networks can be trained on massive amounts of data and find patterns that are not included in existing theories. The challenge is to distill these insights from the neural networks and incorporate them into scientific understanding.<\/p>\n\n\n\n<p>Cranmer proposes symbolic distillation as a method to achieve this. This method involves interpreting a neural network by finding a set of equations that replicates the network\u2019s behavior. These equations can then be analyzed to understand the underlying scientific principles learned by the network.<\/p>\n\n\n\n<p>Cranmer acknowledges that there are limitations to symbolic distillation, particularly its inability to find complex symbolic models. However, he believes that this is an exciting direction for scientific discovery, especially with the development of foundation models like <strong><a href=\"https:\/\/polymathic-ai.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Polymathic AI<\/a><\/strong>. These models are trained on massive datasets across many disciplines and are likely to contain broadly applicable scientific models.<\/p>\n\n\n\n<p>The talk concludes with a discussion of open questions. One question is how to incorporate more general algorithms, beyond simple mathematical equations, into symbolic regression. Another question is whether pre-training neural networks on certain types of data can be detrimental to scientific discovery. Cranmer acknowledges that there are likely to be adversarial examples, where pre-training on certain data sets can make it harder to find scientific insights, but so far such examples haven\u2019t been discovered.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Miles Cranmer\u662f\u5251\u6865\u5927\u5b66\u52a9\u7406\u6559\u6388\uff0c\u4ed6\u4e8e2024\u5e744\u6708\u5728Simons Foundation\u53d1\u8868\u7684\u6f14\u8bb2 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[36],"tags":[39,97,66],"class_list":["post-4121","post","type-post","status-publish","format-standard","hentry","category-36","tag-ai","tag-distillation","tag-google"],"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"NullThought","author_link":"https:\/\/nullthought.net\/?author=1"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/nullthought.net\/?cat=36\" rel=\"category\">\u79d1\u5b66<\/a>","rttpg_excerpt":"Miles Cranmer\u662f\u5251\u6865\u5927\u5b66\u52a9\u7406\u6559\u6388\uff0c\u4ed6\u4e8e2024\u5e744\u6708\u5728Simons Foundation\u53d1\u8868\u7684\u6f14\u8bb2&hellip;","_links":{"self":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/4121","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4121"}],"version-history":[{"count":6,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/4121\/revisions"}],"predecessor-version":[{"id":4133,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/4121\/revisions\/4133"}],"wp:attachment":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}