NCA-GENL practice question 12 of 15
Which of the following prompt engineering techniques is most effective for improving an LLM's performance on multi-step reasoning tasks?
Full set: 95 questionsDomain Large Language Models (LLMs)
Chain-of-thought (CoT) prompting is a technique specifically designed to improve the reasoning abilities of Large Language Models (LLMs) on complex, multi-step tasks. By providing the model with examples that include explicit, intermediate reasoning steps, CoT guides the model to break down a problem into a sequence of logical thoughts before arriving at a final answer. This process mimics human-like reasoning and has been shown to significantly enhance performance on arithmetic, commonsense, and symbolic reasoning problems that require multiple inferential steps.
A. Retrieval-augmented generation without context: Retrieval-augmented generation (RAG) is inherently dependent on retrieving and using external context to inform the model's response. RAG without context is a contradiction and would not function as intended.
B. Few-shot prompting with unrelated examples: The effectiveness of few-shot prompting relies on providing relevant examples that demonstrate the desired task format and logic. Unrelated examples would confuse the model and degrade its performance.
C. Zero-shot prompting with detailed task descriptions: While a detailed description is helpful, zero-shot prompting is often insufficient for complex reasoning tasks, as it does not explicitly show the model how to perform the intermediate reasoning steps.
1. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv preprint arXiv:2201.11903. (Section 2, "Chain-of-Thought Prompting," describes how providing a series of intermediate reasoning steps significantly improves performance on multi-step arithmetic and symbolic reasoning tasks). https://doi.org/10.48550/arXiv.2201.11903
2. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv preprint arXiv:2005.11401. (Section 2.1, "The RAG model," explains that the core mechanism involves a retriever finding relevant documents to be used as context). https://doi.org/10.48550/arXiv.2005.11401
3. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language Models are Few-Shot Learners. arXiv preprint arXiv:2005.14165. (Section 2.2, "Few-shot," demonstrates that performance scales with the number of relevant examples provided in the prompt's context). https://doi.org/10.48550/arXiv.2005.14165
4. Zhao, Z., et al. (2023). A Survey of Large Language Models. arXiv preprint arXiv:2303.18223. (Section 4.2.1, "Chain-of-Thought (CoT) Prompting," explicitly states that CoT is a key technique for enhancing the reasoning capabilities of LLMs by generating intermediate steps). https://doi.org/10.48550/arXiv.2303.18223
Keep practicing
Take the free NCA-GENL practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other NCA-GENL practice questions
- Question 1What is the purpose of few-shot learning in prompt engineering?
- Question 2Which model deployment framework is used to deploy an NLP project, especially for high- p…
- Question 3Why do we need positional encoding in transformer-based models?
- Question 4What is Retrieval Augmented Generation (RAG)?
- Question 5Which technology will allow you to deploy an LLM for production application?
- Question 6In evaluating the transformer model for translation tasks, what is a common approach to a…
