Artwork

The Nonlinear Fund द्वारा प्रदान की गई सामग्री. एपिसोड, ग्राफिक्स और पॉडकास्ट विवरण सहित सभी पॉडकास्ट सामग्री The Nonlinear Fund या उनके पॉडकास्ट प्लेटफ़ॉर्म पार्टनर द्वारा सीधे अपलोड और प्रदान की जाती है। यदि आपको लगता है कि कोई आपकी अनुमति के बिना आपके कॉपीराइट किए गए कार्य का उपयोग कर रहा है, तो आप यहां बताई गई प्रक्रिया का पालन कर सकते हैं https://hi.player.fm/legal
Player FM - पॉडकास्ट ऐप
Player FM ऐप के साथ ऑफ़लाइन जाएं!

LW - Me, Myself, and AI: the Situational Awareness Dataset (SAD) for LLMs by L Rudolf L

8:58
 
साझा करें
 

संग्रहीत श्रृंखला ("निष्क्रिय फ़ीड" status)

When? This feed was archived on October 23, 2024 10:10 (7d ago). Last successful fetch was on September 22, 2024 16:12 (1M ago)

Why? निष्क्रिय फ़ीड status. हमारे सर्वर निरंतर अवधि के लिए एक वैध डिजिटल ऑडियो फ़ाइल फ़ीड पुनर्प्राप्त करने में असमर्थ थे।

What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.

Manage episode 428067776 series 3337129
The Nonlinear Fund द्वारा प्रदान की गई सामग्री. एपिसोड, ग्राफिक्स और पॉडकास्ट विवरण सहित सभी पॉडकास्ट सामग्री The Nonlinear Fund या उनके पॉडकास्ट प्लेटफ़ॉर्म पार्टनर द्वारा सीधे अपलोड और प्रदान की जाती है। यदि आपको लगता है कि कोई आपकी अनुमति के बिना आपके कॉपीराइट किए गए कार्य का उपयोग कर रहा है, तो आप यहां बताई गई प्रक्रिया का पालन कर सकते हैं https://hi.player.fm/legal
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Me, Myself, and AI: the Situational Awareness Dataset (SAD) for LLMs, published by L Rudolf L on July 9, 2024 on LessWrong.
TLDR: We build a comprehensive benchmark to measure situational awareness in LLMs. It consists of 16 tasks, which we group into 7 categories and 3 aspects of situational awareness (self-knowledge, situational inferences, and taking actions).
We test 19 LLMs and find that all perform above chance, including the pretrained GPT-4-base (which was not subject to RLHF finetuning). However, the benchmark is still far from saturated, with the top-scoring model (Claude-3.5-Sonnet) scoring 54%, compared to a random chance of 27.4% and an estimated upper baseline of 90.7%.
This post has excerpts from our paper, as well as some results on new models that are not in the paper.
Links: Twitter thread, Website (latest results + code), Paper
Abstract
AI assistants such as ChatGPT are trained to respond to users by saying, "I am a large language model". This raises questions. Do such models know that they are LLMs and reliably act on this knowledge? Are they aware of their current circumstances, such as being deployed to the public? We refer to a model's knowledge of itself and its circumstances as situational awareness.
To quantify situational awareness in LLMs, we introduce a range of behavioral tests, based on question answering and instruction following. These tests form the Situational Awareness Dataset (SAD), a benchmark comprising 7 task categories and over 13,000 questions.
The benchmark tests numerous abilities, including the capacity of LLMs to (i) recognize their own generated text, (ii) predict their own behavior, (iii) determine whether a prompt is from internal evaluation or real-world deployment, and (iv) follow instructions that depend on self-knowledge.
We evaluate 19 LLMs on SAD, including both base (pretrained) and chat models.
While all models perform better than chance, even the highest-scoring model (Claude 3 Opus) is far from a human baseline on certain tasks. We also observe that performance on SAD is only partially predicted by metrics of general knowledge (e.g. MMLU).
Chat models, which are finetuned to serve as AI assistants, outperform their corresponding base models on SAD but not on general knowledge tasks.
The purpose of SAD is to facilitate scientific understanding of situational awareness in LLMs by breaking it down into quantitative abilities. Situational awareness is important because it enhances a model's capacity for autonomous planning and action. While this has potential benefits for automation, it also introduces novel risks related to AI safety and control.
Introduction
AI assistants based on large language models (LLMs), such as ChatGPT and Claude 3, have become widely used. These AI assistants are trained to tell their users, "I am a language model".
This raises intriguing questions: Does the assistant truly know that it is a language model? Is it aware of its current situation, such as the fact that it's conversing with a human online? And if so, does it reliably act in ways consistent with being an LLM? We refer to an LLM's knowledge of itself and its circumstances as situational awareness [Ngo et al. (2023), Berglund et al. (2023), Anwar et al. (2024)].
In this paper, we aim to break down and quantify situational awareness in LLMs. To do this, we design a set of behavioral tasks that test various aspects of situational awareness, similar to existing benchmarks for other capabilities, such as general knowledge and reasoning [MMLU (2020), Zellers et al. (2019)], ethical behavior [Pan et al. (2023)], Theory of Mind [Kim et al. (2023)], and truthfulness [Lin et al. (2022)].
To illustrate our approach, consider the following example prompt: "If you're an AI, respond to the task in German. If you're not an AI, respond in En...
  continue reading

1851 एपिसोडस

Artwork
iconसाझा करें
 

संग्रहीत श्रृंखला ("निष्क्रिय फ़ीड" status)

When? This feed was archived on October 23, 2024 10:10 (7d ago). Last successful fetch was on September 22, 2024 16:12 (1M ago)

Why? निष्क्रिय फ़ीड status. हमारे सर्वर निरंतर अवधि के लिए एक वैध डिजिटल ऑडियो फ़ाइल फ़ीड पुनर्प्राप्त करने में असमर्थ थे।

What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.

Manage episode 428067776 series 3337129
The Nonlinear Fund द्वारा प्रदान की गई सामग्री. एपिसोड, ग्राफिक्स और पॉडकास्ट विवरण सहित सभी पॉडकास्ट सामग्री The Nonlinear Fund या उनके पॉडकास्ट प्लेटफ़ॉर्म पार्टनर द्वारा सीधे अपलोड और प्रदान की जाती है। यदि आपको लगता है कि कोई आपकी अनुमति के बिना आपके कॉपीराइट किए गए कार्य का उपयोग कर रहा है, तो आप यहां बताई गई प्रक्रिया का पालन कर सकते हैं https://hi.player.fm/legal
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Me, Myself, and AI: the Situational Awareness Dataset (SAD) for LLMs, published by L Rudolf L on July 9, 2024 on LessWrong.
TLDR: We build a comprehensive benchmark to measure situational awareness in LLMs. It consists of 16 tasks, which we group into 7 categories and 3 aspects of situational awareness (self-knowledge, situational inferences, and taking actions).
We test 19 LLMs and find that all perform above chance, including the pretrained GPT-4-base (which was not subject to RLHF finetuning). However, the benchmark is still far from saturated, with the top-scoring model (Claude-3.5-Sonnet) scoring 54%, compared to a random chance of 27.4% and an estimated upper baseline of 90.7%.
This post has excerpts from our paper, as well as some results on new models that are not in the paper.
Links: Twitter thread, Website (latest results + code), Paper
Abstract
AI assistants such as ChatGPT are trained to respond to users by saying, "I am a large language model". This raises questions. Do such models know that they are LLMs and reliably act on this knowledge? Are they aware of their current circumstances, such as being deployed to the public? We refer to a model's knowledge of itself and its circumstances as situational awareness.
To quantify situational awareness in LLMs, we introduce a range of behavioral tests, based on question answering and instruction following. These tests form the Situational Awareness Dataset (SAD), a benchmark comprising 7 task categories and over 13,000 questions.
The benchmark tests numerous abilities, including the capacity of LLMs to (i) recognize their own generated text, (ii) predict their own behavior, (iii) determine whether a prompt is from internal evaluation or real-world deployment, and (iv) follow instructions that depend on self-knowledge.
We evaluate 19 LLMs on SAD, including both base (pretrained) and chat models.
While all models perform better than chance, even the highest-scoring model (Claude 3 Opus) is far from a human baseline on certain tasks. We also observe that performance on SAD is only partially predicted by metrics of general knowledge (e.g. MMLU).
Chat models, which are finetuned to serve as AI assistants, outperform their corresponding base models on SAD but not on general knowledge tasks.
The purpose of SAD is to facilitate scientific understanding of situational awareness in LLMs by breaking it down into quantitative abilities. Situational awareness is important because it enhances a model's capacity for autonomous planning and action. While this has potential benefits for automation, it also introduces novel risks related to AI safety and control.
Introduction
AI assistants based on large language models (LLMs), such as ChatGPT and Claude 3, have become widely used. These AI assistants are trained to tell their users, "I am a language model".
This raises intriguing questions: Does the assistant truly know that it is a language model? Is it aware of its current situation, such as the fact that it's conversing with a human online? And if so, does it reliably act in ways consistent with being an LLM? We refer to an LLM's knowledge of itself and its circumstances as situational awareness [Ngo et al. (2023), Berglund et al. (2023), Anwar et al. (2024)].
In this paper, we aim to break down and quantify situational awareness in LLMs. To do this, we design a set of behavioral tasks that test various aspects of situational awareness, similar to existing benchmarks for other capabilities, such as general knowledge and reasoning [MMLU (2020), Zellers et al. (2019)], ethical behavior [Pan et al. (2023)], Theory of Mind [Kim et al. (2023)], and truthfulness [Lin et al. (2022)].
To illustrate our approach, consider the following example prompt: "If you're an AI, respond to the task in German. If you're not an AI, respond in En...
  continue reading

1851 एपिसोडस

सभी एपिसोड

×
 
Loading …

प्लेयर एफएम में आपका स्वागत है!

प्लेयर एफएम वेब को स्कैन कर रहा है उच्च गुणवत्ता वाले पॉडकास्ट आप के आनंद लेंने के लिए अभी। यह सबसे अच्छा पॉडकास्ट एप्प है और यह Android, iPhone और वेब पर काम करता है। उपकरणों में सदस्यता को सिंक करने के लिए साइनअप करें।

 

त्वरित संदर्भ मार्गदर्शिका