Scientia · Science
AI science
Papers, concepts and people behind AI: 198 in this graph, including AlexNet, AlphaFold, AlphaGo, BERT, ResNet and T5.
Capabilities in this galaxy
Models12
- AlexNet2012The convolutional network that won ImageNet 2012 by a wide margin and sparked the deep learning boom.Runs offline
- AlphaFold2018DeepMind's system for predicting protein structures from amino acid sequences.Open source, free
- AlphaGo2016DeepMind's Go program that beat Lee Sedol in 2016 using deep networks and tree search.
- BERT2018Google's bidirectional Transformer encoder that set the pretrain-then-fine-tune pattern in NLP.Open source, free · Runs offline
- ResNet2015Microsoft Research's residual network, whose skip connections made very deep networks trainable.Open source, free · Runs offline
- T52019Google's Text-to-Text Transfer Transformer, which cast every NLP task as text in, text out.Open source, free · Runs offline
- AlphaZero2017DeepMind's self-taught system that mastered chess, shogi and Go without human game data.
- DQN2013DeepMind's Deep Q-Network, which learned to play Atari games from raw pixels.
- word2vec2013Google's method and tool for learning word embeddings from large text corpora.Open source, free · Runs offline
- Alpaca2023Stanford's instruction-following model fine-tuned from LLaMA on self-instruct data.Open source, free · Runs offline
- ELMo2018Ai2's contextual word embeddings from a bidirectional LSTM language model.Open source, free · Runs offline
- Genie2024Google DeepMind's family of world models that generate interactive, playable environments.
Organizations16
- Google DeepMind2010Google's AI lab, founded as DeepMind in 2010 and merged with Google Brain in 2023; it builds Gemini.
- Carnegie Mellon University1900A Pittsburgh research university that created the first academic machine learning department.
- MIT1959A research university in Massachusetts whose AI Project, founded in 1959, grew into CSAIL.
- Stanford University1963A private research university in California whose AI Lab (SAIL) was founded in 1963.
- Tsinghua University 清华大学1911A research university in Beijing whose AI groups co-developed the GLM models.
- UC BerkeleyA public research university in California and home of the Berkeley AI Research (BAIR) lab.
- University of Toronto2012A Canadian research university where Geoffrey Hinton's group built AlexNet in 2012.
- Ai22014The Allen Institute for AI, a Seattle nonprofit research lab that releases fully open models.
- EleutherAI2020A grassroots nonprofit research group that released The Pile and early open GPT-style models.
- LAION2021A German nonprofit that builds open datasets such as LAION-5B for training multimodal models.
- Mila1993A Montreal AI research institute founded by Yoshua Bengio in 1993.
- New York University2013A research university in New York City whose Center for Data Science was founded in 2013 under Yann LeCun.
- Princeton University1985A research university in New Jersey where the WordNet database began in 1985 and ImageNet began.
- Safe Superintelligence2024An AI research company co-founded by Ilya Sutskever to build safe superintelligence.
- Thinking Machines Lab2025An AI research and product company founded in 2025 by former OpenAI CTO Mira Murati.
- Technology Innovation Institute2020An Abu Dhabi government research institute that builds the Falcon language models.
People33
- Alan Turing1950British mathematician who formalized computation and proposed the Turing test for machine intelligence.
- Demis Hassabis2010Co-founder of DeepMind who led AlphaGo and AlphaFold; shared the 2024 Nobel Prize in Chemistry.
- Geoffrey Hinton1986Deep learning pioneer known for backpropagation, deep belief nets and AlexNet; Turing Award and Nobel laureate.
- Yann LeCun1989Deep learning pioneer who developed convolutional neural networks for image recognition; Turing Award laureate.
- Andrej Karpathy2015AI researcher and educator, a founding member of OpenAI, who coined the term vibe coding.
- Andrew Ng2012Researcher and educator who co-founded Google Brain and Coursera and popularized online machine learning courses.
- Fei-Fei Li2009Computer scientist who created the ImageNet dataset, which helped spark the deep learning revolution.
- Ian Goodfellow2014Researcher who invented generative adversarial networks and co-wrote the textbook Deep Learning.
- Ilya Sutskever2012Researcher who co-created AlexNet and sequence-to-sequence learning and co-founded OpenAI and SSI.
- Jeff Dean2012Google engineer and researcher who co-founded Google Brain and co-designed large-scale systems such as MapReduce.
- John McCarthy1955Computer scientist who coined the term artificial intelligence and created the Lisp programming language.
- Kaiming He2015Computer vision researcher who introduced ResNet and its residual connections.
- Marvin Minsky1955AI pioneer who co-organized the 1956 Dartmouth workshop and co-founded the MIT AI Lab.
- Richard Sutton1988Reinforcement learning pioneer who developed temporal-difference learning; 2024 Turing Award laureate.
- Yoshua Bengio1993Deep learning pioneer who founded Mila and advanced neural language models and attention; Turing Award laureate.
- Alec Radford2018Researcher who was lead author of OpenAI's GPT, GPT-2 and CLIP papers.
- Alex Krizhevsky2009Researcher who built AlexNet, the GPU-trained network that won ImageNet 2012, and created CIFAR-10.
- Ashish Vaswani2017First author of Attention Is All You Need, the paper that introduced the Transformer.
- Dan Hendrycks2020Researcher who created the MMLU and MATH benchmarks and led Humanity's Last Exam.
- David Silver2016Reinforcement learning researcher who led DeepMind's AlphaGo and AlphaZero projects.
- François Chollet2019Creator of the Keras deep learning library and the ARC-AGI benchmark.
- Frank Rosenblatt1958Psychologist who invented the perceptron, one of the first trainable neural networks.
- Jared Kaplan2020Physicist turned AI researcher who led OpenAI's scaling-laws study and co-founded Anthropic.
- Jeremy Howard2018Co-founder of fast.ai, co-author of ULMFiT and proposer of the llms.txt standard.
- John Jumper2021Researcher who led the development of AlphaFold 2; shared the 2024 Nobel Prize in Chemistry.
- Jonathan Ho2020Researcher who introduced denoising diffusion probabilistic models (DDPM).
- Jürgen Schmidhuber1997Computer scientist who co-invented the LSTM recurrent neural network with Sepp Hochreiter.
- Noam Shazeer2017Co-author of the Transformer paper and pioneer of sparse mixture-of-experts models; co-founded Character.AI.
- Oriol Vinyals2014Researcher who co-authored sequence-to-sequence learning and knowledge distillation and led AlphaStar.
- Quoc V. Le2012Google researcher who co-authored sequence-to-sequence learning and pioneered neural architecture search.
- Sepp Hochreiter1997Computer scientist who analyzed the vanishing gradient problem and co-invented LSTM.
- Tomáš Mikolov2013Researcher who created word2vec, popularizing learned word embeddings.
- Paul Christiano2017Alignment researcher who co-developed reinforcement learning from human feedback (RLHF).
Concepts69
- AI Agent1995A system that perceives its environment and acts toward goals, today often an LLM calling tools in a loop.
- Artificial General Intelligence (AGI)2007Hypothetical AI that matches or exceeds humans across most cognitive tasks.
- Artificial Intelligence1956The field of building computer systems that perform tasks associated with human intelligence.
- Deep Learning2006Machine learning with neural networks of many layers that learn representations directly from raw data.
- Generative AI2022AI systems that create new text, images, audio, video or code rather than only classifying data.
- Large Language Model (LLM)2018A large neural network trained on vast text to predict the next token, able to follow instructions and converse.
- Machine Learning1959A branch of AI in which systems learn patterns from data instead of following hand-written rules.
- Neural Network1943A model made of layers of connected artificial neurons whose weights are learned from data.
- Transformer2017A neural network architecture built on attention that underlies nearly all modern language models.
- AI Alignment2016Research and techniques for making AI systems act in line with human intentions and values.
- Attention2014A mechanism that lets a model weigh the most relevant parts of its input when producing each output.
- Backpropagation1986The algorithm that trains neural networks by passing error gradients backward through the layers.
- Chain-of-Thought2022Having a model write out intermediate reasoning steps before its final answer.
- Code Generation2021Using AI models to write, complete or edit source code from natural language or context.
- Context Window2018The maximum amount of text, measured in tokens, that a model can consider at once.
- Convolutional Neural Network (CNN)1989A neural network that applies learned convolution filters, widely used for images.
- Diffusion Model2015A generative model that learns to create data by reversing a gradual noising process.
- Embeddings2013Dense numeric vectors that represent words, text, images or other data so similar items lie close together.
- Fine-Tuning2014Further training a pretrained model on a smaller, specific dataset to adapt it to a task.
- Function Calling2023A model outputting structured calls to external tools or APIs that an application then executes.
- Generative Adversarial Network (GAN)2014A generative model in which a generator and a discriminator network are trained against each other.
- Hallucination2018When a model states false or unsupported information as if it were fact.
- Long Short-Term Memory (LSTM)1997A recurrent network with gated memory cells that can learn long-range dependencies in sequences.
- LoRA2021A fine-tuning method that trains small low-rank matrices added to a frozen model.
- Mixture of Experts (MoE)1991An architecture that routes each input to a few specialized sub-networks, so only part of the model runs.
- Multimodality2021The ability of a model to take in or produce more than one kind of data, such as text, images and audio.
- Open Weights2023Releasing a trained model's weights publicly so anyone can download and run it.
- Pretraining2018The first, large-scale training stage in which a model learns general knowledge from broad data.
- Prompt Engineering2020Designing and refining the instructions and examples given to a model to get better outputs.
- Reasoning Model2024A language model trained to think through a problem at length before answering.
- Reinforcement Learning1988Learning by trial and error, where an agent improves its actions to maximize a reward signal.
- Retrieval-Augmented Generation (RAG)2020Improving a model's answers by retrieving relevant documents and adding them to its prompt.
- RLHF2017Fine-tuning a model with reinforcement learning against a reward model trained on human preferences.
- Scaling Laws2020Empirical rules showing model loss falls predictably as parameters, data and compute grow.
- Speech Recognition2012Converting spoken audio into written text.
- Text-to-Image2021Generating images from a written description.
- Text-to-Speech2016Converting written text into natural-sounding spoken audio.
- Text-to-Video2022Generating video clips from a written description.
- Tokenization2015Splitting text into tokens, such as subword units, that a language model reads and writes.
- Turing Test1950Turing's test of whether a machine's conversation can be told apart from a human's.
- Vibe Coding2025Building software by describing what you want to an AI and accepting its code with little manual review.
- Computer Use2024An AI model operating a computer by reading the screen and controlling the mouse and keyboard.
- Constitutional AI2022An Anthropic method that trains a model to follow a written set of principles using AI feedback.
- Context Engineering2025Deciding what information, tools and history go into a model's context window for each step of a task.
- DPO2023A preference-tuning method that optimizes a model directly on preference pairs without a separate reward model.
- FlashAttention2022An IO-aware algorithm that computes exact attention faster and with less GPU memory.
- In-Context Learning2020A model's ability to pick up a task from instructions or examples in its prompt without weight updates.
- Inference2016Running a trained model to produce outputs, as opposed to training it.
- Knowledge Distillation2015Training a smaller student model to imitate the outputs of a larger teacher model.
- Knowledge Graph2012A network of entities and the typed relations between them, used to store and query knowledge.
- Latent Diffusion2021Diffusion performed in the compressed latent space of an autoencoder, making high-resolution generation cheaper.
- Multi-Agent System1995A system in which several agents interact, cooperate or compete to solve a problem.
- Music Generation2020Using AI models to compose music or produce songs, often from a text prompt.
- Perceptron1958An early single-layer neural network that learns a linear classifier by adjusting its weights.
- Prompt Injection2022An attack in which untrusted text hijacks a model by smuggling in instructions that override the developer's.
- Quantization2015Storing and running a model's weights at lower numeric precision to save memory and compute.
- ReAct2022A prompting pattern in which a language model alternates reasoning steps with actions such as tool calls.
- Recurrent Neural Network (RNN)1986A neural network with loops that processes sequences one step at a time while keeping a hidden state.
- Residual Connection2015A shortcut that adds a layer's input to its output, making very deep networks trainable.
- Self-Supervised Learning2018Learning from unlabeled data by predicting parts of the input from other parts.
- Semantic Search2019Search that matches meaning rather than exact keywords, usually by comparing embeddings.
- Sequence-to-Sequence (Seq2Seq)2014An encoder–decoder approach that maps one sequence to another, first applied to machine translation.
- Vector Database2019A database that stores embedding vectors and finds the nearest ones to a query quickly.
- Vision Transformer (ViT)2020A Transformer that processes images by treating fixed-size patches as tokens.
- Voice Cloning2018Generating speech in a specific person's voice from a short sample of their recordings.
- World Model2018An internal model of how an environment works that an AI uses to predict and plan.
- Image Upscaling2014Increasing an image's resolution while adding plausible detail, also called super-resolution.
- Ontology1993A formal specification of the classes, properties and relations used to describe a domain.
- Temporal-Difference Learning1988A reinforcement learning method that updates value estimates from the difference between successive predictions.
Papers45
- Attention Is All You Need2017Paper introducing the Transformer, an architecture based entirely on attention.
- Computing Machinery and Intelligence1950Alan Turing's paper asking whether machines can think and proposing the imitation game, later called the Turing test.
- ImageNet Classification with Deep Convolutional Neural Networks2012Paper presenting AlexNet, the GPU-trained CNN whose ImageNet 2012 win launched the deep learning boom.
- A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence1955The 1955 proposal for the 1956 Dartmouth workshop, which coined the term artificial intelligence.
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale2020Paper introducing the Vision Transformer, which applies a standard Transformer to image patches.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018Google paper introducing BERT, a bidirectional Transformer pretrained with masked language modeling.
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models2022Google paper showing that prompting with step-by-step reasoning examples improves LLM reasoning.
- Deep Residual Learning for Image Recognition2015Paper introducing residual networks, whose skip connections made networks over 100 layers deep trainable.
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025DeepSeek paper showing reasoning abilities can be developed in LLMs through large-scale reinforcement learning.
- Denoising Diffusion Probabilistic Models2020Paper showing denoising diffusion models can generate high-quality images.
- Efficient Estimation of Word Representations in Vector Space2013Paper introducing word2vec's efficient architectures for learning word vectors from large text corpora.
- Generative Adversarial Nets2014Paper introducing generative adversarial networks, where a generator and a discriminator compete.
- Gradient-Based Learning Applied to Document Recognition1998Paper presenting the LeNet-5 convolutional network for handwriting recognition and the MNIST dataset.
- High-Resolution Image Synthesis with Latent Diffusion Models2021Paper introducing latent diffusion models, the approach behind Stable Diffusion.
- Highly Accurate Protein Structure Prediction with AlphaFold2021Nature paper describing AlphaFold 2, which predicts protein 3D structures with high accuracy.
- ImageNet: A Large-Scale Hierarchical Image Database2009Paper introducing ImageNet, a large image database organized by the WordNet hierarchy.
- Improving Language Understanding by Generative Pre-Training2018OpenAI paper introducing GPT: generative pretraining of a Transformer followed by task fine-tuning.
- Language Models are Few-Shot Learners2020OpenAI paper introducing GPT-3 and showing large language models can learn tasks from a few examples in the prompt.
- Learning Representations by Back-Propagating Errors1986Nature paper showing that backpropagation lets multilayer networks learn useful internal representations.
- Learning Transferable Visual Models From Natural Language Supervision2021OpenAI paper introducing CLIP, which learns a joint image-text embedding from web image-caption pairs.
- LLaMA: Open and Efficient Foundation Language Models2023Meta paper introducing LLaMA, a family of foundation language models trained on public data.
- Long Short-Term Memory1997Paper introducing the LSTM recurrent network, designed to overcome vanishing gradients.
- LoRA: Low-Rank Adaptation of Large Language Models2021Microsoft paper introducing LoRA, which fine-tunes large models by training small low-rank update matrices.
- Mastering the Game of Go with Deep Neural Networks and Tree Search2016Nature paper describing AlphaGo, which combined deep neural networks with tree search to master Go.
- Neural Machine Translation by Jointly Learning to Align and Translate2014Paper introducing the attention mechanism for neural machine translation.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks2020Paper introducing retrieval-augmented generation, combining a retriever with a sequence-to-sequence generator.
- Scaling Laws for Neural Language Models2020OpenAI paper finding power-law relationships between language model loss and model size, data and compute.
- The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain1958Rosenblatt's paper describing the perceptron, a trainable model inspired by neurons.
- Training Language Models to Follow Instructions with Human Feedback2022OpenAI paper introducing InstructGPT, fine-tuned with human feedback to follow instructions.
- Constitutional AI: Harmlessness from AI Feedback2022Anthropic paper introducing Constitutional AI, training a harmless assistant from AI feedback guided by principles.
- Deep Reinforcement Learning from Human Preferences2017Paper showing agents can learn complex behaviors from human comparisons of their actions, the basis of RLHF.
- Deep Unsupervised Learning using Nonequilibrium Thermodynamics2015Paper introducing diffusion probabilistic models, which learn to reverse a gradual noising process.
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model2023Stanford paper introducing Direct Preference Optimization, aligning models to preferences without RL.
- Distilling the Knowledge in a Neural Network2015Paper introducing knowledge distillation, transferring a large model's knowledge into a smaller one.
- Evaluating Large Language Models Trained on Code2021OpenAI paper introducing Codex and the HumanEval benchmark for writing Python functions from docstrings.
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness2022Paper introducing FlashAttention, an IO-aware exact attention algorithm that speeds up Transformers.
- Language Models are Unsupervised Multitask Learners2019OpenAI paper introducing GPT-2 and showing language models can perform tasks without task-specific training.
- ReAct: Synergizing Reasoning and Acting in Language Models2022Paper introducing ReAct, in which language models interleave reasoning traces with actions.
- Sequence to Sequence Learning with Neural Networks2014Paper showing that LSTM encoder–decoder networks can translate one sequence into another end to end.
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity2021Google paper introducing the Switch Transformer, a sparse mixture-of-experts model routing each token to one expert.
- Toolformer: Language Models Can Teach Themselves to Use Tools2023Meta paper in which a language model teaches itself when and how to call external tools.
- Training Compute-Optimal Large Language Models2022DeepMind paper finding that model size and training tokens should grow together for compute-optimal training.
- Universal Language Model Fine-tuning for Text Classification2018Paper introducing ULMFiT, showing a pretrained language model can be fine-tuned for text classification.
- Adaptive Mixtures of Local Experts1991Paper introducing a system of expert networks with a gating network that decides which expert handles each input.
- Building High-level Features Using Large Scale Unsupervised Learning2012Google paper in which a large network trained on unlabeled YouTube frames learned to detect cats and faces.
Benchmarks14
- ARC-AGI2019François Chollet's grid-puzzle benchmark measuring how efficiently a system learns new skills from few examples.Open source, free
- Arena (LMArena)2023A public leaderboard that ranks AI models by crowdsourced head-to-head votes, started as Chatbot Arena.
- HumanEval2021OpenAI's set of 164 hand-written Python programming problems checked by unit tests.Open source, free
- Humanity's Last Exam2025A benchmark of expert-written questions across many subjects, built to be hard for frontier models.
- MMLU2020Multiple-choice questions across 57 subjects, from math to law, testing a model's knowledge and reasoning.Open source, free
- SWE-bench2023A benchmark that asks AI systems to resolve real GitHub issues from Python repositories, checked by tests.Open source, free
- BIG-bench2022A collaborative benchmark of over 200 diverse tasks meant to probe the capabilities of language models.Open source, free
- GLUE2018A collection of nine English language-understanding tasks used to compare pretrained language models.Open source, free
- GPQA2023Graduate-level multiple-choice questions in biology, physics and chemistry written to be hard to look up.Open source, free
- GSM8K2021About 8,500 grade-school math word problems that require multi-step arithmetic reasoning.Open source, free
- MATH202112,500 competition mathematics problems with step-by-step solutions.Open source, free
- MMMU2023College-level multimodal questions across many disciplines that combine images with text.Open source, free
- SQuAD2016Stanford's reading-comprehension dataset of questions whose answers are spans of Wikipedia passages.Open source, free
- TruthfulQA2021Questions designed to test whether a model repeats common human misconceptions instead of true answers.Open source, free
Datasets9
- ImageNet2009A database of over 14 million labeled images organized by WordNet, whose challenge drove the deep learning boom.
- Common Crawl2007A nonprofit's free, open archive of web crawl data, a major source of text for training language models.Open source, free
- MNIST1998A dataset of 70,000 labeled images of handwritten digits, a classic starting point for machine learning.Open source, free
- C42019Google's cleaned English web-text corpus derived from Common Crawl, introduced to train the T5 model.Open source, free
- CIFAR-10200960,000 small color images in 10 classes, a standard dataset for image classification research.Open source, free
- COCO2014Microsoft's dataset of everyday scenes annotated for object detection, segmentation and captioning.Open source, free
- FineWeb2024Hugging Face's large, filtered English web-text dataset built from Common Crawl for LLM pretraining.Open source, free
- LAION-5B2022An open dataset of about 5.85 billion image-text pairs gathered from the web, used to train image models.
- The Pile2020EleutherAI's roughly 800 GB English text corpus combining 22 diverse sources for training language models.Open source, free
Last updated 2026-09-24