A person provides the content of an email to an AI model and asks it to identify whether the email is a promotion. The person prompts the model…

Self-consistency is an advanced prompting technique used to enhance the accuracy of an LLM's reasoning. It involves prompting the model with the same question multiple times to generate several different reasoning paths or outputs. The final answer is then determined by a majority vote from the varied responses. This method leverages the idea that a correct answer is more likely to be reached through multiple lines of reasoning. The scenario of prompting repeatedly and taking the most frequent response perfectly describes the self-consistency method.

A. Chain-of-Thought: This involves instructing the model to think step-by-step in a single prompt, not running the same prompt multiple times.

B. Iterative Prompting: This involves changing the prompt to improve the result, whereas self-consistency uses the same prompt repeatedly.

C. Few-shot Prompting: This provides examples within a prompt to guide the model, which is different from sampling multiple outputs.

1. Wang, X., et al. (2023). "Self-Consistency Improves Chain of Thought Reasoning in Language Models." arXiv preprint arXiv:2203.11171. Section 2, "Self-Consistency," defines the method as sampling diverse reasoning paths and selecting the most consistent answer by marginalizing out the reasoning paths. (https://doi.org/10.48550/arXiv.2203.11171)

2. Wei, J., et al. (2023). "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." arXiv preprint arXiv:2201.11903. Section 5.2 discusses how self-consistency can be used as a decoding strategy to improve upon chain-of-thought prompting. (https://doi.org/10.48550/arXiv.2201.11903)