Survey Papers List

Quick links

|||| |—|—|—| | 🚀 General | 🧱 Prompting Techniques | 🩸Small Language Models | | 🟩 Multi Modal LLMs| 🚧 MUltilingual LLMs | 📤 Data Management | | 🌠 Post Training | 💎 LLM Fine Tuning|⚖️ LLM Inference | | 🔍 LLM Architectures | 📅 LLM Abilities and Limitations | 📝 LLM Reasoning | | 🛑 Chain of Thought | 💠 LLM RAG | ❇️ LLM Agents | | ✒️ Data Augmentation | 🎈 Hallucination | 🟢 Fairness | |✅ Embedding Models| ⚓️ LLM Evaluation| 📝 Synthetic Data Generation| | 🔴 LLM Safety | 🔶 LLMs for Specific Domains|☑️ Others|

Related Repositories

• 🧱LLM Engineer Toolkit - Contains a curated list of 120+ LLM libraries category wise.
• 🚀RAG Zero to Hero Guide - Comprehensive guide to learn RAG from basics to advanced.

General

| Paper | Link | |——-|——| | Large Language Models: A Survey | Link | | A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4 | Link | | A Comprehensive Overview of Large Language Models | Link | | A Survey of Large Language Models | Link | | Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects | Link |

Prompting Techniques

| Paper | Link | |——-|——| | The Prompt Report: A Systematic Survey of Prompting Techniques | Link | | Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization | Link | | Efficient Prompting Methods for Large Language Models: A Survey | Link | | A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications | Link | | In-context Learning with Retrieved Demonstrations for Language Models: A Survey | Link |

Small Language Models

| Paper | Link | |——-|——| | Small Language Models (SLMs) Can Still Pack a Punch: A survey | Link | | A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness | Link | | A Survey of Small Language Models | Link | | Small Language Models: Survey, Measurements, and Insights | Link |

Multi Modal LLMs

| Paper | Link | |——-|——| | Surveying the MLLM Landscape: A Meta-Review of Current Surveys | Link | | From Specific-MLLM to Omni-MLLM: A Survey about the MLLMs alligned with Multi-Modality | Link | | How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model | Link | | Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey | Link | | MM-LLMs: Recent Advances in MultiModal Large Language Models | Link | | A Survey on Multimodal Large Language Models | Link |

Multilingual LLMs

| Paper | Link | |——-|——| | Multilingual Large Language Models: A Systematic Survey | Link | | A Survey of Large Language Models for European Languages | Link | | A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers | Link | | Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers | Link | | A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias | Link |

Data Management

| Paper | Link | |——-|——| | A Survey on Data Selection for Language Models | Link | | Datasets for Large Language Models: A Comprehensive Survey | Link | | A Survey on Data Selection for LLM Instruction Tuning | Link | | Data Management For Training Large Language Models: A Survey | Link |

Post Training

| Paper | Link | |——-|——| | A Survey on Post-training of Large Language Models | Link | | LLM Post-Training: A Deep Dive into Reasoning Large Language Models | Link | | Reinforcement Learning Enhanced LLMs: A Survey | Link | | Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models | Link | | A Comprehensive Survey of LLM Alignment Techniques: RLHF, RLAIF, PPO, DPO and More | Link | | Towards Scalable Automated Alignment of LLMs: A Survey | Link | | Large Language Model Alignment: A Survey | Link | | Instruction Tuning for Large Language Models: A Survey | Link |

LLM Fine-Tuning

| Paper | Link | |——-|——| | Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies | Link | | Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey | Link | | Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment | Link |

LLM Inference

| Paper | Link | |——-|——| | A Survey on LLM Inference-Time Self-Improvement | Link | | A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models | Link | | Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems | Link | | A Thorough Examination of Decoding Methods in the Era of LLMs | Link | | Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward | Link |

LLM Architectures

| Paper | Link | |——-|——| | A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications | Link | | A Survey on Mixture of Experts | Link |

LLM Abilities and Limitations

| Paper | Link | |——-|——| | Emergent Abilities in Large Language Models: A Survey | Link | | A Survey on Multi-Turn Interaction Capabilities of Large Language Models | Link | | A Survey on Large Language Models with some Insights on their Capabilities and Limitations | Link | | Knowledge Boundary of Large Language Models: A Survey | Link | | A Survey on the Honesty of Large Language Models | Link |

LLM Reasoning

| Paper | Link | |——-|——| | A Survey on Enhancing Causal Reasoning Ability of Large Language Models | Link | | Reasoning Language Models: A Blueprint | Link | | Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models | Link | | A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges | Link | | A Survey of Table Reasoning with Large Language Models | Link | | Large Language Models for Mathematical Reasoning: Progresses and Challenges | Link | | Towards Reasoning in Large Language Models: A Survey | Link |

Chain of Thought

| Paper | Link | |——-|——| | Igniting Language Intelligence: The Hitchhiker’s Guide From Chain-of-Thought Reasoning to Language Agents | Link | | Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future | Link | | Towards Better Chain-of-Thought Prompting Strategies: A Survey | Link |

LLM RAG

| Paper | Link | |——-|——| | Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG | Link | | A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models | Link | | Exploring AI Text Generation, Retrieval-Augmented Generation, and Detection Technologies: a Comprehensive Overview | Link | | Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely | Link | | Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey | Link | | Trustworthiness in Retrieval-Augmented Generation Systems: A Survey | Link | | Graph Retrieval-Augmented Generation: A Survey | Link | | Evaluation of Retrieval-Augmented Generation: A Survey | Link | | A Survey on Knowledge-Oriented Retrieval-Augmented Generation | Link | | Retrieval-Augmented Generation for AI-Generated Content: A Survey | Link |

LLM Agents

| Paper | Link | |——-|——| | Survey on Evaluation of LLM-based Agents | Link | | GUI Agents: A Survey | Link | | Large Language Model-Brained GUI Agents: A Survey | Link | | GUI Agents with Foundation Models: A Comprehensive Survey | Link | | The Rise and Potential of Large Language Model Based Agents: A Survey | Link | | A Survey on Large Language Model-based Agents for Statistics and Data Science | Link | | Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents | Link | | Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems | Link | | A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios | Link | | From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents | Link | | Agents in Software Engineering: Survey, Landscape, and Vision | Link | | Large Language Model-Based Agents for Software Engineering: A Survey | Link | | From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future | Link | | Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security | Link | | Large Language Model Agent in Financial Trading: A Survey | Link | | Tool Learning with Large Language Models: A Survey | Link | | What Are Tools Anyway? A Survey from the Language Model Perspective | Link | | Large Language Model based Multi-Agents: A Survey of Progress and Challenges | Link | | Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects | Link |

Data Augmentation

| Paper | Link | |——-|——| | Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunities | Link | | Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges | Link | | A Survey on Data Augmentation in Large Model Era | Link |

Hallucination

| Paper | Link | |——-|——| | A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions | Link | | A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models | Link | | Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey | Link |

Fairness

| Paper | Link | |——-|——| | Fairness Definitions in Language Models Explained | Link | | A survey on fairness of large language models in e-commerce: progress, application, and challenge | Link | | Fairness in Large Language Models: A Taxonomic Survey | Link | | A Survey on Fairness in Large Language Models | Link |

Embedding Models

| Paper | Link | |——-|——| | LLMs are Also Effective Embedding Models: An In-depth Overview | Link | | When Text Embedding Meets Large Language Model: A Comprehensive Survey | Link |

LLM Evaluation

| Paper | Link | |——-|——| | LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods | Link | | From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge | Link | | A Survey on LLM-as-a-Judge | Link | | Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices | Link | | A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations | Link | | The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches | Link | | A Survey of Useful LLM Evaluation | Link | | Leveraging Large Language Models for NLG Evaluation: A Survey | Link | | Evaluating Large Language Models: A Comprehensive Survey | Link | | Through the Lens of Core Competency: Survey on Evaluation of Large Language Models | Link | | A Survey on Evaluation of Large Language Models | Link |

Synthetic Data Generation

| Paper | Link | |——-|——| | Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models | Link | | On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey | Link | | Synthetic Data Generation Using Large Language Models: Advances in Text and Code | Link | | A Survey on Recent Advances in Conversational Data Generation | Link |

LLM Safety

| Paper | Link | |——-|——| | Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations | Link | | AI Safety in Generative AI Large Language Models: A Survey | Link | | Jailbreak Attacks and Defenses Against Large Language Models: A Survey | Link | | Against The Achilles’ Heel: A Survey on Red Teaming for Generative Models | Link | | On Protecting the Data Privacy of Large Language Models (LLMs): A Survey | Link | | Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models | Link | | Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey | Link | | Building Guardrails for Large Language Models | Link | | Security and Privacy Challenges of Large Language Models: A Survey | Link | | A Security Risk Taxonomy for Large Language Models | Link | | A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation | Link |

LLMs for Specific Domains

| Paper | Domain | Link | |——-|——–|——| | Large language models (LLM) in computational social science: prospects, current state, and challenges | Social Media | Link | | A Survey on Large Language Models for Code Generation | Coding | Link | | Large Language Models Meet NL2Code: A Survey | Coding | Link | | Large Language Models for Education: A Survey | Education | Link | | Political-LLM: Large Language Models in Political Science | Political Science | Link | | Code LLMs: A Taxonomy-based Survey | Coding | Link | | Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey | Data Science | Link | | A Survey of Large Language Models in Finance (FinLLMs) | Finance | Link | | Large Language Models in Finance: A Survey | Finance | Link | | Revolutionizing Finance with LLMs: An Overview of Applications and Insights | Finance | Link | | Legal Evalutions and Challenges of Large Language Models | Legal | Link | | Large Language Models in Law: A Survey | Legal | Link | | A Survey for Large Language Models in Biomedicine | Healthcare | Link | | A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences | Healthcare | Link | | Demystifying Large Language Models for Medicine: A Primer | Healthcare | Link | | Large Language Models for Disease Diagnosis: A Scoping Review | Healthcare | Link | | A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations | Healthcare | Link | | A Survey of Large Language Models in Medicine: Principles, Applications, and Challenges | Healthcare | Link | | Large Language Models for Medicine: A Survey | Healthcare | Link | | Large Language Models in Biomedical and Health Informatics: A Bibliometric Review | Healthcare | Link | | A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions | Healthcare | Link | | A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics | Healthcare | Link | | AI for Biomedicine in the Era of Large Language Models | Healthcare | Link | | Large Language Models in Bioinformatics: A Survey | Healthcare | Link | | Large language models for automated scholarly paper review: A survey | Scientific Research | Link | | LLM4SR: A Survey on Large Language Models for Scientific Research | Scientific Research | Link | | A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery | Scientific Research | Link | | Harnessing Large Language Models for Disaster Management: A Survey | Disaster Management | Link | | Leveraging Foundation Models for Crafting Narrative Visualization: A Survey | Visualization | Link | | LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights | Security | Link | | Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey | Security | Link | | A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions | Security | Link | | Mathematical Language Models: A Survey | Maths | Link | | Software Testing with Large Language Model: Survey, Landscape, and Vision | Software Engineering | Link | | Large Language Models for Software Engineering: Survey and Open Problems | Software Engineering | Link |

Others

| Paper | Category | Link | |——-|———-|——| | Awes, Laws, and Flaws From Today’s LLM Research | LLM Research | Link | | Efficient Large Language Models: A Survey | Efficient LLMs | Link | | Continual Learning for Large Language Models: A Survey | Continual Learning | Link | | From Selection to Generation: A Survey of LLM-based Active Learning | Active Learning | Link | | Towards Lifelong Learning of Large Language Models: A Survey | Life Long Learning | Link | | Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions | Privacy Preserving Mechanism | | | A Survey on Human Preference Learning for Large Language Models | Human Preference Learning | Link | | A Survey on Human-Centric LLMs | Human centric LLMs | Link | | Prompt Compression for Large Language Models: A Survey | Prompts | Link | | The Oscars of AI Theater: A Survey on Role-Playing with Language Models | Prompts | Link | | From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems | Text-to-SQL | Link | | Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL | Text-to-SQL | Link | | A Survey of NL2SQL with Large Language Models: Where are we, and where are we going? | Text-to-SQL | Link | | A Survey on Employing Large Language Models for Text-to-SQL Tasks | Text-to-SQL | Link | | Undesirable Memorization in Large Language Models: A Survey | LLM Memorization | Link | | A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms | Low bit LLMs | Link | | Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges | Data Contamination | Link | | Benchmark Data Contamination of Large Language Models: A Survey | Data Contamination | Link | | Attention Heads of Large Language Models: A Survey | LLM Attention Heads | Link | | Claim Verification in the Age of Large Language Models: A Survey | Fact Checking | Link | | Generative Large Language Models in Automated Fact-Checking: A Survey | Fact Checking | Link | | Controllable Text Generation for Large Language Models: A Survey | Controllable Text Generation | Link | | A Survey on Symbolic Knowledge Distillation of Large Language Models | Knowledge Distillation | Link | | Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application | Knowledge Distillation | Link | | Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities | Model Merging | Link | | Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions | Privacy Preserving | Link | | Keep the Cost Down: A Review on Methods to Optimize LLM’ s KV-Cache Consumption | LLM’s KV-Cache | Link | | Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models | Collaborative Strategies | Link | | When Search Engine Services meet Large Language Models: Visions and Challenges | Search Engines meet LLMs | Link | | A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models | Text Summarization | Link | | LLMs Meet Multimodal Generation and Editing: A Survey | Multi-modal Generation | Link | | Large Language Models on Tabular Data - A Survey | Tabular Data | Link | | Large Language Model for Table Processing: A Survey | Tabular Data | Link | | Large Language Models for Time Series: A Survey | Time Series Data | Link | | A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems | Dialogue Systems | Link | | Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques | Graph Representation Learning | Link | | When Large Language Models Meet Vector Databases: A Survey | Vector Databases | Link | | Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models | Context Length | Link | | The What, Why, and How of Context Length Extension Techniques in Large Language Models – A Detailed Survey | Context Length | Link | | Leveraging Large Models for Crafting Narrative Visualization: A Survey | Visualization | Link | | Large Language Models for Social Networks : Applications, Challenges and Solutions | Social Networks | Link | | Large Language Models for Generative Information Extraction: A Survey | Information Extraction | Link | | A Survey of Text Watermarking in the Era of Large Language Models | Water Marking | Link | | Large Language Models on Graphs: A Comprehensive Survey | Graphs | Link | | A Survey of Graph Meets Large Language Model: Progress and Future Directions | Graphs | Link | | A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions | LLM Text Detection | Link | | A Survey on Detection of LLMs-Generated Content | LLM Text Detection | Link | | Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text | LLM Text Detection | Link | | Large Language Models for Generative Recommendation: A Survey and Visionary Discussions | Recommendations | Link | | Large Language Models for Information Retrieval: A Survey | Information Retrieval | Link | | A Survey on Model Compression for Large Language Models | Model Compression | Link | | A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges | Vision LLMs | Link | | Knowledge Editing for Large Language Models: A Survey | Knowledge Editing | Link | | A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods | Knowledge Bases | Link |