Named Entity Recognition: How NER Works and Where It Fits
Named entity recognition explained: what NER extracts, how dictionary, rule-based and BERT models work, how to score them, and how it applies to business documents.
Named entity recognition (NER) is the task of finding the names in unstructured text and labeling each one with a type, such as person, organization, location, date or amount. The input is a block of text. The output is that same text annotated with entities and their categories, ready for search, analytics or automation. This guide covers how NER works, the main approaches, how to measure it, and what it means when your text lives in invoices, contracts and forms.
TL;DR
- NER is a subtask of information extraction: it locates spans of text that name things and classifies them into predefined categories.
- Approaches fall into four families: dictionary-based, rule-based, machine learning and hybrid. Modern systems mostly use machine learning, including transformer models such as BERT.
- Entity labels depend on the task. People, organizations and locations are the common defaults, while finance, medicine and law add their own types.
- Score NER at the entity level and say which matching rule you used (strict, exact, partial or type), or your numbers cannot be compared.
- For business documents, recognizing names is one step. Validating fields and pushing them into your systems is what removes manual entry.
Table of Contents
- What named entity recognition does
- Which entity types NER can find
- How NER works
- Using BERT for named entity recognition
- How to evaluate NER performance
- NER versus named entity resolution
- NER on business documents
- Our perspective
- Choosing an approach for your documents
- See it on your own documents
- FAQ
- Sources
What named entity recognition does
NER is a subtask of information extraction that locates named entities in unstructured text and sorts them into predefined categories. It is also called entity identification, entity chunking or entity extraction. The standard illustration turns the sentence “Jim bought 300 shares of Acme Corp. in 2006” into a version where Jim is tagged as a person, Acme Corp. as an organization and 2006 as a time.
A named entity is, in the words of the Stanford speech and language processing text, roughly anything that can be referred to with a proper name. In practice the term is stretched to cover dates, times and numerical expressions such as prices. NER therefore has two jobs: find the span of words that forms the name, then decide its type.
Why does that matter? A 2025 review in Artificial Intelligence Review describes NER as crucial for information extraction and as a support for downstream work such as event extraction, summarization, relation extraction, question answering and machine translation. Almost every system that turns text into structured data leans on it somewhere.
Which entity types NER can find
There is no single universal list. The Stanford chapter notes that entity types are task-specific: people, places and organizations are common, while gene or protein names and financial asset classes matter in other tasks.
The generic set it lists looks like this:
| Label | Meaning | Example from the source |
|---|---|---|
| PER | Person | Turing is a giant of computer science |
| ORG | Organization | The IPCC warned about the cyclone |
| LOC | Location | Mt. Sanitas is in Canyon |
| GPE | Geo-political entity | Palo Alto is raising the fees for parking |
| FAC | Facility | Consider the Golden Gate Bridge |
| VEH | Vehicle | It was a classic Ford Falcon |
Other tag sets are smaller. The Universal NER annotation guidelines define only person, organization and location, plus an OTHER tag for spans that feel like entities but fit none of them.
The same words can change type with context. The same guidelines point out that “The White House” is a location when something happens there and an organization when it decides something. The Seow review adds that an entity in one domain may be a non-entity in another. Define your label set before you choose a model, because the labels decide what “correct” means.
Labels in business text
In commercial documents the useful entities are often the ones general tag sets lack: vendor names, invoice numbers, purchase order references, part numbers, payment terms, effective dates, party names in a contract. IBM’s overview lists monetary values, quantities, times and medical codes among the categories NER can identify, which shows how far the idea reaches beyond people and places.
How NER works
The Coursera overview describes a typical pipeline of five steps: tokenization, identification, classification, contextual analysis and post-processing. First the text is split into tokens. Then candidate spans are identified, each is assigned a type, context is used to resolve ambiguity, and the result is cleaned up.
Most systems label every word. The Johns Hopkins Applied Physics Laboratory paper explains that most approaches tag each word with its class, using a “not-an-entity” tag for words that are not names. In the common BIO scheme, B marks the beginning of an entity, I marks the inside, and O marks everything outside.
The four families of approach
Dictionary-based. The system looks words up in a list of known names. It is simple and fast, but the European Data Protection Board notes that lexicon-based methods can struggle with new entities and with ambiguous meanings or spelling variants.
Rule-based. Engineers write handcrafted linguistic rules and regular expressions. According to Baeldung, this is the older style, and it works well when the pattern is predictable, such as a fixed invoice number format.
Machine learning. Models learn from human-annotated examples. Baeldung lists hidden Markov models among the classic probabilistic methods, and neural systems built on recurrent networks (including LSTMs) and transformers. The EDPB observes that supervised methods need a significant amount of annotated training data.
Hybrid. Systems combine the above, for example a model for open-ended names and rules for strictly formatted identifiers.
The EDPB also notes that emerging systems using large language models such as BERT, GPT-4, LlaMA and Mistral could reduce the time and cost of annotating training data. That is the direction the field is moving.
Using BERT for named entity recognition
BERT is a common starting point because it can be fine-tuned for token classification. A public example is bert-base-NER on Hugging Face: a bert-base-cased model fine-tuned on the English CoNLL-2003 dataset to recognize four types, location, organization, person and miscellaneous.
The workflow has a familiar shape:
- Choose a cased BERT checkpoint, since capitalization is a strong clue for names.
- Prepare training data in BIO format, with one label per token.
- Align labels to subword tokens. MachineLearningMastery notes that subword tokens may be combined with the previous token at output.
- Add a token classification head and fine-tune.
- Evaluate at the entity level, not only per token.
Read the limits on the model card before reusing a pretrained checkpoint. It was trained on entity-annotated news articles from a specific span of time, so it may not generalize to every domain. A model trained on news will not know your part numbers or your contract clauses unless you teach it with examples from your own documents.
How to evaluate NER performance
Precision, recall and F1 are the standard measures. Precision is the share of predicted entities that are right, recall is the share of true entities you found, and F1 combines them. The subtlety is what counts as a match.
The AI Practitioner Handbook notes that models are often tuned on token-level scores, while downstream evaluation may benefit from aggregating tokens into full entities and comparing them with gold annotations. It describes four SemEval options:
| Scheme | What must match |
|---|---|
| Strict | Exact boundaries and entity type |
| Exact | Exact boundaries, any type |
| Partial | Partial boundary overlap, any type |
| Type | Some overlap, with the correct type |
For invoices and contracts, strict is usually the honest choice. A date with one digit cut off is wrong, not nearly right.
Reporting matters too. Palen-Michel, Holley and Lignos propose guidelines for reproducible evaluation: state the chunk encoding scheme, use and report an external scorer, describe how invalid label sequences are repaired, and apply sound statistics. They found that few papers give enough detail to tell whether those guidelines were followed. When you compare vendors or models, ask for scores computed the same way on the same documents.
NER versus named entity resolution
People also ask about named entity resolution, which is a different step. NER finds a name and labels its type. Resolution decides which real-world thing the name refers to.
The Johns Hopkins paper frames three problems in processing names: recognizing the presence, extent and type of names, linking separate references to the same entity within one document, and linking references across multiple documents. The first is NER. The other two are coreference resolution, within and across documents.
Here is a business example. NER can tag “Acme Corp.”, “Acme Corporation” and “ACME” as organizations. Resolution decides that all three are the same vendor record in your ERP. You need both for clean data.
NER on business documents
Invoices, purchase orders, bills of lading and contracts are not flowing prose. They are semi-structured: tables, headers, stamps, scanned pages with layouts that change by supplier. That affects how you use NER.
Text first. NER works on text. A scanned PDF has to be read into text with its layout before any entity can be tagged. Practitioners who work with technical drawings describe this as a pipeline problem, and this OCR pipeline for drawings shows how much the steps before extraction shape the result.
Fields, not just names. A finance team does not want “Acme Corp.” highlighted. It wants vendor, invoice number, line items, tax and total in a structured record, checked against a schema. That asks for more than entity tags: relationships between values, validation rules and a destination system.
Variation. The EDPB’s point about spelling variants and new entities applies directly. Every new supplier brings a new layout and new naming habits. Approaches that depend on fixed templates or fixed dictionaries need upkeep as that variety grows.
A review step. Where accuracy matters, such as audits, claims or compliance records, a human reviewer should see low-confidence fields with the source text beside them.
For a wider view of extraction beyond NER, see our guide to AI data extraction for business professionals and our technical guide to information retrieval.
Our perspective
NER solved a real problem: it taught machines to find the who, what, where and how much in text. For most of our customers, though, the goal was never a highlighted sentence. The goal is a clean record in the ERP, with no one retyping it.
That is why we built DocuPOW as an agentic document intelligence system rather than a tagger. Our agents are designed to read a document the way a human expert would, including layouts they have never seen, without a template to build or maintain. They validate what they extract against a strict schema and pass the result into your core systems through API.
We also believe people should stay in the loop where it counts. Our human-in-the-loop audit review lets a reviewer confirm a field against its source, and our conversational querying shows citations back to the document, so every answer can be traced.
If your team is weighing entity extraction for invoices, contracts or claims, start from the outcome: which fields, which systems, which checks. Then pick the tooling that gets you there with the least upkeep.
Syed Naveed Abbas
Choosing an approach for your documents
Use these questions to decide.
- What is the output? If you need highlighted names for search or analytics, entity tagging may be enough. If you need validated fields in a system of record, plan for extraction, validation and integration.
- How varied are your sources? One internal format suits rules or a fine-tuned model. Hundreds of supplier layouts favor approaches that do not depend on templates.
- Do you have labeled data? Supervised models need annotated examples, a cost the EDPB flags. LLM-based methods may lower it.
- How will you measure success? Choose strict entity-level scoring, a fixed test set and a target error rate before you start.
- Who reviews exceptions? Decide the confidence threshold and the person who handles anything below it.
- Where does the data go? Map every extracted field to its destination before you build.
See it on your own documents
DocuPOW pairs template-free zero-shot extraction with schema validation, review and ERP and CRM integration. Plans are published: Starter is $99 per month and Professional is $499 per month, with Enterprise available on request. See the pricing page for what each includes.
If invoices are your biggest queue, our page on invoice data extraction for finance teams shows how errors are caught before posting. If your team needs to ask questions of documents, read how document question answering works.
FAQ
What is named entity recognition in simple terms?
It is a way for software to find names in text and label what kind of thing each one is. A system reading “Jim bought shares of Acme Corp. in 2006” marks Jim as a person, Acme Corp. as an organization and 2006 as a time. The result is structured information you can search, count or feed into other systems.
What are the main types of NER models?
The four families are dictionary-based, rule-based, machine learning and hybrid. Dictionary and rule systems are simple but rigid. Machine learning, including transformers such as BERT, learns patterns from annotated examples and now dominates, according to the EDPB.
How do you compute NER accuracy?
Compare predicted entities with a human-labeled gold set and calculate precision, recall and F1. Decide the matching rule first: strict (boundaries and type), exact, partial or type. Report which scorer you used so others can reproduce your numbers.
Can BERT be used for NER?
Yes. You fine-tune a cased BERT model for token classification using BIO labels, aligning labels to subword tokens. Public checkpoints trained on CoNLL-2003 recognize person, organization, location and miscellaneous. For specialized documents, fine-tune on examples from your own domain.
Is NER the same as extracting data from invoices?
Not quite. NER tags names, dates and amounts in text. Invoice processing also needs layout reading, line-item tables, validation against a schema and delivery into an accounting system. NER can be one component of that chain.
Sources
- Named-entity recognition overview
- Information Extraction, Stanford speech and language processing chapter
- A review of named entity recognition, Artificial Intelligence Review
- What Is Named Entity Recognition? IBM
- Annotation Guidelines, Universal NER
- What Is Named Entity Recognition (NER), Coursera
- AI Possible Risks and Mitigations: Named Entity Recognition, EDPB
- Processing Named Entities in Text, JHU Applied Physics Laboratory
- Named Entity Recognition, Baeldung on Computer Science
- bert-base-NER-ONNX model card, Hugging Face
- A Complete Introduction to Using BERT Models, MachineLearningMastery
- What are some metrics for Named Entity Recognition? AI Practitioner Handbook
- SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation
- OCR for drawings: the pipeline that actually works
Recommended
See DocuPOW on your documents.
Stop building templates. Start extracting data.
