How Chain of Thought Prompting Enhances LLM Reasoning by Enabling Serial Computation ?

This research paper explores the relationship between the use of chain of thought (CoT) prompting and the computational complexity of large language models (LLMs). It argues that CoT prompting, where LLMs are instructed to generate intermediate reasoning steps, significantly enhances their ability to solve problems that are inherently serial in nature. The authors analyze the expressiveness of different transformer architectures with varying embedding sizes and CoT lengths, demonstrating that even low-depth transformers can achieve impressive performance on complex tasks when equipped with CoT prompting. Their findings suggest that CoT prompting is a powerful technique for boosting the reasoning capabilities of LLMs, especially for problems that are challenging to parallelize.