HomeAI News
Augmented Language Model Inference: Anthropic's Breakthrough Research

Augmented Language Model Inference: Anthropic's Breakthrough Research

Hayo News
Hayo News
July 19th, 2023
View OriginalTranslated by Google

Anthropic addresses the fidelity challenge in understanding the "improvised reasoning" of language models. The tricky part is determining whether the inferences provided accurately reflect the process the model actually took to make its predictions. Anthropic's research looks at measuring and enhancing the fidelity of reasoning stated by language models, providing valuable insight into their interpretability and reliability.

Anthropic explores possible infidelity by experimenting with modifications to the model's Chain of Thought (CoT) reasoning. For example, they introduce errors during CoT generation to test hypotheses and observe how the model's final answer is affected. These investigations reveal the accuracy and reliability of CoT inference in language models.

In some tasks, Anthropic's experiments show that when the model is restricted to only provide a truncated version of the chain of thought (CoT) to answer the question, it often arrives at different answers. This suggests that CoT is not just a rationalizing explanation. Likewise, when errors are introduced in CoT, the final answer of the model is affected, further supporting the important influence of CoT on the decision-making process of the model. These findings highlight the importance of understanding and improving the fidelity of reasoning stated by language models.

Anthropic investigated whether the so-called "Chain of Thought" (CoT) leads to improved performance simply because of longer inputs (left), or whether it is due to the wording of specific information encoded during inference (right) ) due to. Through their research, they present evidence to refute both hypotheses. These findings challenge previous assumptions and contribute to a deeper understanding of the role and influence of "chain of thought" in language model performance.

Anthropic's findings reveal an interesting phenomenon regarding the inference accuracy of language models. They observed an inverse scaling pattern, whereby as models grow in size and power, their inference accuracy tends to decrease for most research tasks. In cases where inference accuracy is critical, the study suggests that smaller models may be beneficial. These insights have important implications for model selection and interpretation, providing valuable guidance for optimizing performance and reliability on language tasks.

In a study conducted by Radhakrishnan et al., Anthropic explored alternative ways to enhance the accuracy of model generative inference. They investigate two methods for eliciting inferences from models through problem decomposition. This process involves breaking down questions into smaller subquestions that help generate more accurate and comprehensive responses to the original question. By studying these techniques, Anthropic aims to contribute to the development of language models' reasoning capabilities, ultimately improving their interpretability and reliability.

The factorization approach forces the model to generate subquestions, but the model answers the subquestions in a separate context. And the "chain of thought" decomposition method also lets the model generate subquestions, but answer all subquestions in a single context, just like the "chain of thought" prompt.

These problem decomposition methods help to mark points on a spectrum with "chain of thought" cues at one end, factorization at the other end, and "chain of thought" decomposition at the link in the middle.

Decomposition alleviates the problem of models ignoring their reasoning process by explicitly specifying the relationships between inference steps. Answering subquestions in isolated contexts can also reduce the model's ability to generate biased inferences.

Anthropic's research revealed a notable finding that language models tend to rely more on factorization-based reasoning. Using a metric proposed by Lanham et al., they evaluate the model's behavior when forced to answer with a truncated or corrupted version of factorized inference. From the analysis, Anthropic concluded that under these conditions, the responses of the models changed more significantly, indicating the importance of disaggregated reasoning in their decision-making processes. These findings help understand how language models use inference strategies and provide insights to improve their accuracy and reliability.

Anthropic's survey reveals a clear trade-off between question answering accuracy and inference accuracy when using different inference generation methods. Notably, their decomposition-based approach shows the potential to push the performance and accuracy Pareto front, suggesting that there is a trade-off between improved question answering accuracy and more reliable inference. These promising results instill optimism for future advances in this field, motivating further research and development to improve the interpretability and reliability of language models.

Here are two papers published by Anthropic:

Measuring accuracy in "chain of thought" reasoning

Problem decomposition improves the accuracy of model generative inference

Reprinted from AnthropicView Original


no dataCoffee time! Feel free to comment