{"id":3423,"date":"2024-01-07T11:24:37","date_gmt":"2024-01-07T03:24:37","guid":{"rendered":"https:\/\/nullthought.net\/?p=3423"},"modified":"2025-02-23T11:09:47","modified_gmt":"2025-02-23T03:09:47","slug":"transformer%e7%9a%84%e8%87%aa%e6%b3%a8%e6%84%8f%e5%8a%9b%e6%9c%ba%e5%88%b6%ef%bc%88self-attention%ef%bc%89","status":"publish","type":"post","link":"https:\/\/nullthought.net\/?p=3423","title":{"rendered":"\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff08self-attention\uff09\u7684\u80cc\u540e\u662f\u201c\u7406\u6027\u201d"},"content":{"rendered":"\n<p>Transformer\u7684\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff08self-attention\uff09\uff0c\u5176\u80cc\u540e\u662f\u7406\u6027\u548c\u903b\u8f91\u3002\u65f6\u5e8f\u6570\u636e\u4e2d\u5982\u679c\u5929\u7136\u9690\u542b\u7406\u6027\u8def\u5f84\uff0c\u6b64\u65f6\u5728AI\u6a21\u578b\u8bbe\u8ba1\u5f15\u5165\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff0c\u53ef\u80fd\u4f1a\u6709\u66f4\u597d\u7684\u7ed3\u679c\u8f93\u51fa\u3002\u6bd4\u5982\u8fd9\u4e2a<a href=\"https:\/\/www.nature.com\/articles\/s41551-023-01010-8\" target=\"_blank\" rel=\"noreferrer noopener\">\u673a\u5668\u4eba\u8f85\u52a9\u5916\u79d1\u624b\u672f\u7684\u89c6\u9891\u5206\u6790AI\u5efa\u6a21<\/a>\u5f15\u5165self-attention\uff0c\u5176\u524d\u63d0\u662f\u89c4\u8303\u3001\u4f18\u79c0\u7684\u624b\u672f\u8fc7\u7a0b\u5185\u542b\u201c\u7406\u6027\u201d\u3002<br><\/p>\n\n\n\n<p>The self-attention mechanism of the Transformer is rooted in rationality and logic. If time-series data inherently follows a rational path, introducing self-attention into AI model design could potentially improve outcomes. For instance in <a href=\"https:\/\/www.nature.com\/articles\/s41551-023-01010-8\" target=\"_blank\" rel=\"noreferrer noopener\">this paper<\/a>, incorporating self-attention in AI models for video analysis of robot-assisted surgeries presupposes that the surgery process, being standardized and exemplary, embodies &#8220;rationality.&#8221;<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"792\" data-id=\"3425\" src=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-1024x792.jpg\" alt=\"\" class=\"wp-image-3425\" srcset=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-1024x792.jpg 1024w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-300x232.jpg 300w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-768x594.jpg 768w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos.jpg 1100w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"467\" data-id=\"3424\" src=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-self-attention.jpg\" alt=\"\" class=\"wp-image-3424\" srcset=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-self-attention.jpg 1000w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-self-attention-300x140.jpg 300w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-self-attention-768x359.jpg 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"923\" height=\"1024\" data-id=\"3426\" src=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-figure-1-923x1024.jpg\" alt=\"\" class=\"wp-image-3426\" srcset=\"https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-figure-1-923x1024.jpg 923w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-figure-1-270x300.jpg 270w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-figure-1-768x852.jpg 768w, https:\/\/nullthought.net\/wp-content\/uploads\/2024\/01\/A-vision-transformer-for-decoding-surgeon-activity-from-surgical-videos-figure-1.jpg 1080w\" sizes=\"auto, (max-width: 923px) 100vw, 923px\" \/><\/figure>\n<figcaption class=\"blocks-gallery-caption wp-element-caption\"><strong>A vision transformer for decoding surgeon activity from surgical videos<\/strong><br>authors: Dani Kiyasseh, etc., Source: <a href=\"https:\/\/www.nature.com\/articles\/s41551-023-01010-8\" target=\"_blank\" rel=\"noreferrer noopener\">nature.com<\/a><\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Transformer\u7684\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff08self-attention\uff09\uff0c\u5176\u80cc\u540e\u662f\u7406\u6027\u548c\u903b\u8f91\u3002\u65f6\u5e8f\u6570\u636e\u4e2d\u5982\u679c\u5929\u7136\u9690 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[8,36],"tags":[39,95,61],"class_list":["post-3423","post","type-post","status-publish","format-standard","hentry","category-tech","category-36","tag-ai","tag-transformer","tag-61"],"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"NullThought","author_link":"https:\/\/nullthought.net\/?author=1"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/nullthought.net\/?cat=8\" rel=\"category\">Tech<\/a> <a href=\"https:\/\/nullthought.net\/?cat=36\" rel=\"category\">\u79d1\u5b66<\/a>","rttpg_excerpt":"Transformer\u7684\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff08self-attention\uff09\uff0c\u5176\u80cc\u540e\u662f\u7406\u6027\u548c\u903b\u8f91\u3002\u65f6\u5e8f\u6570\u636e\u4e2d\u5982\u679c\u5929\u7136\u9690&hellip;","_links":{"self":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/3423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3423"}],"version-history":[{"count":3,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/3423\/revisions"}],"predecessor-version":[{"id":3429,"href":"https:\/\/nullthought.net\/index.php?rest_route=\/wp\/v2\/posts\/3423\/revisions\/3429"}],"wp:attachment":[{"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nullthought.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}