the open
    research
    Contribute

    2026 edition · Top 100

    Language, speech & LLMs

    Natural language processing, large language models, and speech and audio.

    Large language models: 36 · Natural language processing: 43 · Speech & audio: 21 · N = 100, scaled from 245 candidates.

    0 of 100 of this page's 100 read · saved in this browser only

    1. 1
      Natural language processing2014

      Neural Machine Translation by Jointly Learning to Align and Translate

      Dzmitry Bahdanau · 30,207 citations · Papers With Code

      AwardEditor's note
    2. 2
      Natural language processing2014

      Sequence to Sequence Learning with Neural Networks

      Ilya Sutskever · 22,497 citations · Papers With Code

      AwardEditor's note
    3. 3
      Large language models2017

      Attention Is All You Need

      Ashish Vaswani · 197,211 citations · Papers With Code

      Editor's note
    4. 4
      Natural language processing2014

      GloVe: Global Vectors for Word Representation

      Jeffrey Pennington · 35,748 citations · Semantic Scholar

      Award
    5. 5
      Natural language processing2013

      Efficient Estimation of Word Representations in Vector Space

      Tomas Mikolov · 35,287 citations · Papers With Code

      Editor's note
    6. 6
      Natural language processing2015

      Neural Machine Translation of Rare Words with Subword Units

      Rico Sennrich · 9,118 citations · Papers With Code

      AwardEditor's note
    7. 7
      Large language models2020

      Language Models are Few-Shot Learners

      Tom B. Brown · 64,736 citations · Papers With Code

      Editor's note
    8. 8
      Natural language processing2018

      BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

      Jacob Devlin · 122,869 citations · Papers With Code

      Editor's note
    9. 9
      Natural language processing2016

      SQuAD: 100,000+ Questions for Machine Comprehension of Text

      Pranav Rajpurkar · 9,917 citations · Papers With Code

      Award
    10. 10
      Speech & audio2016

      WaveNet: A Generative Model for Raw Audio

      Aäron van den Oord · 8,473 citations · Papers With Code

      Editor's note
    11. 11
      Large language models2020

      Scaling Laws for Neural Language Models

      Jared Kaplan · 9,325 citations · Papers With Code

      Editor's note
    12. 12
      Large language models2023

      LLaMA: Open and Efficient Foundation Language Models

      Hugo Touvron · 22,056 citations · Papers With Code

      Editor's note
    13. 13
      Natural language processing2019

      Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

      Colin Raffel · 27,780 citations · Papers With Code

      Editor's note
    14. 14
      Large language models2022

      Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

      Jason Wei · 22,892 citations · Papers With Code

      Editor's note
    15. 15
      Speech & audio2022

      Robust Speech Recognition via Large-Scale Weak Supervision

      Alec Radford · 9,010 citations · Papers With Code

      Editor's note
    16. 16
      Large language models2020

      Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

      Patrick Lewis · 19,550 citations · Papers With Code

      Editor's note
    17. 17
      Natural language processing2002

      BLEU: a Method for Automatic Evaluation of Machine Translation

      Kishore Papineni · 21,338 citations · OpenAlex

      Editor's note
    18. 18
      Speech & audio2020

      wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

      Alexei Baevski · 9,551 citations · Papers With Code

      Editor's note
    19. 19
      Natural language processing2018

      Deep contextualized word representations

      Matthew E. Peters · 12,316 citations · Papers With Code

      Editor's note
    20. 20
      Large language models2023

      Direct Preference Optimization: Your Language Model is Secretly a Reward Model

      Rafael Rafailov · 11,135 citations · Papers With Code

      Editor's note
    21. 21
      Natural language processing2001

      Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

      John Lafferty · 15,788 citations · Semantic Scholar

      Editor's note
    22. 22
      Large language models2025

      DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

      DeepSeek-AI · 5,325 citations · Papers With Code

      ContestedEditor's note
    23. 23
      Natural language processing2003

      A Neural Probabilistic Language Model

      Yoshua Bengio · 8,212 citations · Semantic Scholar

      Editor's note
    24. 24
      Large language models2022

      ReAct: Synergizing Reasoning and Acting in Language Models

      Shunyu Yao · 12,540 citations · Papers With Code

      Editor's note
    25. 25
      Large language models2022

      Training Compute-Optimal Large Language Models

      Jordan Hoffmann · 3,889 citations · Papers With Code

      Editor's note
    26. 26
      Speech & audio2006

      Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

      Alex Graves · 5,370 citations · OpenAlex

      Editor's note
    27. 27
      Natural language processing1993

      The Mathematics of Statistical Machine Translation: Parameter Estimation

      Peter F. Brown · 4,957 citations · Semantic Scholar

      Editor's note
    28. 28
      Large language models2023

      GPT-4 Technical Report

      OpenAI · 27,772 citations · Papers With Code

    29. 29
      Large language models2021

      Finetuned Language Models Are Zero-Shot Learners

      Jason Wei · 5,503 citations · Papers With Code

      Editor's note
    30. 30
      Natural language processing1993

      Building a Large Annotated Corpus of English: The Penn Treebank

      Mitchell P. Marcus · 9,337 citations · Semantic Scholar

      Editor's note
    31. 31
      Speech & audio2017

      Tacotron: Towards End-to-End Speech Synthesis

      Yuxuan Wang · 2,110 citations · Papers With Code

      Editor's note
    32. 32
      Natural language processing2018

      GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

      Alex Wang · 9,100 citations · Papers With Code

    33. 33
      Speech & audio2021

      HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

      Wei-Ning Hsu · 5,295 citations · Papers With Code

    34. 34
      Large language models2023

      Llama 2: Open Foundation and Fine-Tuned Chat Models

      Hugo Touvron · 18,620 citations · Papers With Code

    35. 35
      Natural language processing2015

      Effective Approaches to Attention-based Neural Machine Translation

      Thang Luong · 8,497 citations · Papers With Code

      Award
    36. 36
      Large language models2022

      PaLM: Scaling Language Modeling with Pathways

      Aakanksha Chowdhery · 8,476 citations · Papers With Code

    37. 37
      Large language models2020

      Measuring Massive Multitask Language Understanding

      Dan Hendrycks · 10,072 citations · Papers With Code

    38. 38
      Natural language processing2019

      RoBERTa: A Robustly Optimized BERT Pretraining Approach

      Yinhan Liu · 31,842 citations · Papers With Code

    39. 39
      Large language models2023

      Toolformer: Language Models Can Teach Themselves to Use Tools

      Timo Schick · 6,197 citations · Papers With Code

    40. 40
      Large language models2021

      RoFormer: Enhanced Transformer with Rotary Position Embedding

      Jianlin Su · 6,815 citations · Papers With Code

    41. 41
      Natural language processing1996

      A Maximum Entropy Approach to Natural Language Processing

      Adam L. Berger · 3,706 citations · Semantic Scholar

      Award
    42. 42
      Natural language processing2011

      Natural Language Processing (Almost) from Scratch

      Ronan Collobert · 8,134 citations · Semantic Scholar

    43. 43
      Natural language processing2013

      Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

      Richard Socher · 6,876 citations · OpenAlex

      Award
    44. 44
      Large language models2024

      The Llama 3 Herd of Models

      Aaron Grattafiori · 19,899 citations · Papers With Code

    45. 45
      Natural language processing1995

      WordNet: A Lexical Database for English

      George A. Miller · 14,020 citations · OpenAlex

    46. 46
      Speech & audio2015 · published 2016

      Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

      Dario Amodei · 3,204 citations · Papers With Code

    47. 47
      Natural language processing2014

      Convolutional Neural Networks for Sentence Classification

      Yoon Kim · 14,830 citations · OpenAlex

    48. 48
      Large language models2021

      Evaluating Large Language Models Trained on Code

      Mark Chen · 12,149 citations · Papers With Code

    49. 49
      Speech & audio2020

      Conformer: Convolution-augmented Transformer for Speech Recognition

      Anmol Gulati · 4,410 citations · Papers With Code

    50. 50
      Natural language processing2004

      ROUGE: A Package for Automatic Evaluation of Summaries

      Chin-Yew Lin · 22,647 citations · Semantic Scholar

    51. 51
      Natural language processing1999

      Foundations of Statistical Natural Language Processing

      Christopher D. Manning · 9,816 citations · OpenAlex

    52. 52
      Natural language processing2018

      SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

      Taku Kudo · 4,363 citations · Papers With Code

    53. 53
      Natural language processing2014

      Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

      Kyunghyun Cho · 27,927 citations · Papers With Code

    54. 54
      Speech & audio2014

      Deep Speech: Scaling up end-to-end speech recognition

      Awni Hannun · 2,290 citations · Papers With Code

    55. 55
      Natural language processing2016

      Enriching Word Vectors with Subword Information

      Piotr Bojanowski · 11,059 citations · Papers With Code

    56. 56
      Large language models2022

      Self-Consistency Improves Chain of Thought Reasoning in Language Models

      Xuezhi Wang · 7,986 citations · Papers With Code

    57. 57
      Natural language processing2019

      Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

      Nils Reimers · 21,379 citations · Papers With Code

    58. 58
      Natural language processing2019 · published 2020

      BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

      Mike Lewis · 13,369 citations · Papers With Code

    59. 59
      Natural language processing2019

      XLNet: Generalized Autoregressive Pretraining for Language Understanding

      Zhilin Yang · 9,600 citations · Papers With Code

    60. 60
      Large language models2025

      Qwen3 Technical Report

      An Yang · 9,262 citations · Papers With Code

    61. 61
      Natural language processing2019

      ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

      Zhenzhong Lan · 7,796 citations · Papers With Code

    62. 62
      Speech & audio2017 · published 2018

      Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions

      Jonathan Shen · 3,211 citations · Papers With Code

    63. 63
      Large language models2023

      Let's Verify Step by Step

      Hunter Lightman · 5,038 citations · Papers With Code

    64. 64
      Speech & audio2020

      HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

      Jungil Kong · 2,913 citations · Papers With Code

    65. 65
      Large language models2022

      Emergent Abilities of Large Language Models

      Wei · 3,899 citations · Papers With Code

    66. 66
      Natural language processing2003

      Statistical Phrase-Based Translation

      Philipp Koehn · 3,964 citations · Semantic Scholar

    67. 67
      Large language models2023

      Mamba: Linear-Time Sequence Modeling with Selective State Spaces

      Albert Gu · 9,660 citations · Papers With Code

    68. 68
      Speech & audio2018 · published 2019

      Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation

      Yi Luo · 2,281 citations · Papers With Code

    69. 69
      Speech & audio2019

      SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

      Daniel S. Park · 4,294 citations · Papers With Code

    70. 70
      Speech & audio1980

      Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences

      Steven B. Davis · 5,465 citations · OpenAlex

    71. 71
      Natural language processing2016

      Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

      Yonghui Wu · 7,419 citations · Papers With Code

    72. 72
      Natural language processing2018

      Universal Language Model Fine-tuning for Text Classification

      Jeremy Howard · 4,403 citations · Semantic Scholar

    73. 73
      Large language models2023 · published 2024

      Lost in the Middle: How Language Models Use Long Contexts

      Nelson F. Liu · 5,542 citations · Papers With Code

    74. 74
      Large language models2023

      Tree of Thoughts: Deliberate Problem Solving with Large Language Models

      Shunyu Yao · 5,154 citations · Papers With Code

    75. 75
      Natural language processing2007

      Moses: Open Source Toolkit for Statistical Machine Translation

      Philipp Koehn · 6,599 citations · Semantic Scholar

    76. 76
      Speech & audio1978

      Dynamic programming algorithm optimization for spoken word recognition

      Hiroaki Sakoe · 6,707 citations · OpenAlex

    77. 77
      Speech & audio2015

      Librispeech: An ASR corpus based on public domain audio books

      Vassil Panayotov · 6,072 citations · OpenAlex

    78. 78
      Speech & audio2012

      Sequence Transduction with Recurrent Neural Networks

      Alex Graves · 2,159 citations · Papers With Code

    79. 79
      Large language models2023

      SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

      Carlos E. Jimenez · 4,406 citations · Papers With Code

    80. 80
      Speech & audio2021 · published 2022

      WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing

      Sanyuan Chen · 3,817 citations · Papers With Code

    81. 81
      Large language models2021

      Training Verifiers to Solve Math Word Problems

      Karl Cobbe · 11,365 citations · Papers With Code

    82. 82
      Speech & audio1995

      An Information-Maximization Approach to Blind Separation and Blind Deconvolution

      Anthony J. Bell · 8,988 citations · OpenAlex

    83. 83
      Natural language processing2019

      SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

      Wang · 2,989 citations · Papers With Code

    84. 84
      Large language models2022

      Self-Instruct: Aligning Language Models with Self-Generated Instructions

      Yizhong Wang · 3,770 citations · Papers With Code

    85. 85
      Natural language processing2016

      Neural Architectures for Named Entity Recognition

      Guillaume Lample · 4,438 citations · Papers With Code

    86. 86
      Large language models2024

      Mixtral of Experts

      Albert Q. Jiang · 2,214 citations · Papers With Code

    87. 87
      Large language models2023

      Sparks of Artificial General Intelligence: Early Experiments with GPT-4

      Bubeck · 4,512 citations · Papers With Code

    88. 88
      Natural language processing2019

      Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

      Zihang Dai · 4,592 citations · Papers With Code

    89. 89
      Natural language processing2019 · published 2020

      Unsupervised Cross-lingual Representation Learning at Scale

      9,516 citations · Papers With Code

    90. 90
      Large language models2022

      Large Language Models are Zero-Shot Reasoners

      8,234 citations · Papers With Code

    91. 91
      Speech & audio2015

      Listen, Attend and Spell

      William Chan · 302 citations · OpenAlex

    92. 92
      Large language models2021

      The Power of Scale for Parameter-Efficient Prompt Tuning

      Brian Lester · 6,181 citations · Papers With Code

    93. 93
      Natural language processing2016 · published 2017

      Bag of Tricks for Efficient Text Classification

      Armand Joulin · 5,192 citations · Papers With Code

    94. 94
      Natural language processing2014

      The Stanford CoreNLP Natural Language Processing Toolkit

      Christopher Manning · 7,173 citations · OpenAlex

    95. 95
      Speech & audio2019

      wav2vec: Unsupervised Pre-training for Speech Recognition

      Steffen Schneider · 1,290 citations · OpenAlex

    96. 96
      Natural language processing2017

      Get To The Point: Summarization with Pointer-Generator Networks

      Abigail See · 4,570 citations · Papers With Code

    97. 97
      Large language models2022

      Scaling Instruction-Finetuned Language Models

      Hyung Won Chung · 4,514 citations · Papers With Code

    98. 98
      Large language models2021

      Multitask Prompted Training Enables Zero-Shot Task Generalization

      Victor Sanh · 2,087 citations · Papers With Code

    99. 99
      Large language models2021

      Measuring Mathematical Problem Solving With the MATH Dataset

      6,879 citations · Papers With Code

    100. 100
      Natural language processing2008

      A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning

      Ronan Collobert · 5,057 citations · OpenAlex

    Ranked on another page

    Also relevant

    Top 500 papers ranked on another domain page that matter here too.

    1. Computer vision & multimodal2013

      Distributed Representations of Words and Phrases and their Compositionality

      Tomas Mikolov · 35,814 citations · Papers With Code

      AwardEditor's note
    2. Deep learning1997

      Long Short-Term Memory

      Sepp Hochreiter · 102,283 citations · OpenAlex

      Editor's note
    3. Deep learning2021

      LoRA: Low-Rank Adaptation of Large Language Models

      Edward J. Hu · 24,332 citations · Papers With Code

      Editor's note
    4. Safety, alignment & interpretability2022

      Training language models to follow instructions with human feedback

      Long Ouyang · 25,000 citations · Papers With Code

      Editor's note
    5. Machine learning foundations2002 · published 2003

      Latent Dirichlet Allocation

      David M. Blei · 32,337 citations · OpenAlex

      Editor's note
    6. Deep learning2022

      FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

      Tri Dao · 5,579 citations · Papers With Code

      Editor's note
    7. Deep learning2023

      Efficient Memory Management for Large Language Model Serving with PagedAttention

      Woosuk Kwon · 9,202 citations · Papers With Code

      Editor's note
    8. Safety, alignment & interpretability2017

      Deep reinforcement learning from human preferences

      Paul Christiano · 6,187 citations · Papers With Code

      Editor's note
    9. Search, retrieval & recommendation2020

      Dense Passage Retrieval for Open-Domain Question Answering

      Vladimir Karpukhin · 7,327 citations · Papers With Code

      Editor's note
    10. Safety, alignment & interpretability2022

      Constitutional AI: Harmlessness from AI Feedback

      Yuntao Bai · 3,842 citations · Papers With Code

      Editor's note
    11. Computer vision & multimodal2018

      Representation Learning with Contrastive Predictive Coding

      Aaron van den Oord · 14,862 citations · Papers With Code

      Editor's note
    12. Deep learning2023

      QLoRA: Efficient Finetuning of Quantized LLMs

      Tim Dettmers · 6,016 citations · Papers With Code

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0), and Semantic Scholar logoSemantic Scholar (used under its non-commercial API licence). Each count names its source. See the method.