compromise

modest natural language processing

npm install compromise

_{by Spencer Kelly and many contributors}

_{french • german • italian • spanish}

compromise tries its best to turn text into data.
it makes limited and sensible decisions.
_{it's not as smart as you'd think.}

import nlp from 'compromise'
let doc = nlp('she sells seashells by the seashore.')
doc.verbs().toPastTense()
doc.text()
// 'she sold seashells by the seashore.'

if (doc.has('simon says #Verb')) {
  return true
}

let doc = nlp(entireNovel)
doc.match('the #Adjective of times').text()
// "the blurst of times?"

and get data:

import plg from 'compromise-speech'
nlp.extend(plg)
let doc = nlp('Milwaukee has certainly had its share of visitors..')
doc.compute('syllables')
doc.places().json()
/*
[{
  "text": "Milwaukee",
  "terms": [{
    "normal": "milwaukee",
    "syllables": ["mil", "wau", "kee"]
  }]
}]
*/

avoid the problems of brittle parsers:

let doc = nlp("we're not gonna take it..")
doc.has('gonna') // true
doc.has('going to') // true (implicit)
// transform
doc.contractions().expand()
doc.text()
// 'we are not going to take it..'

and whip stuff around like it's data:

let doc = nlp('ninety five thousand and fifty two')
doc.numbers().add(20)
doc.text()
// 'ninety five thousand and seventy two'

_{-because it actually is-}

let doc = nlp('the purple dinosaur')
doc.nouns().toPlural()
doc.text()
// 'the purple dinosaurs'

Use it on the client-side:

<script src="https://unpkg.com/compromise"></script>
<script>
  var doc = nlp('two bottles of beer')
  doc.numbers().minus(1)
  document.body.innerHTML = doc.text()
  // 'one bottle of beer'
</script>

or likewise:

import nlp from 'compromise'
var doc = nlp('London is calling')
doc.verbs().toNegative()
// 'London is not calling'

compromise is ~250kb (minified):

it's pretty fast. It can run on keypress:

it works mainly by conjugating all forms of a basic word list.

The final lexicon is ~14,000 words:

you can read more about how it works, here. it's weird.

_{okay -}

compromise/one

A tokenizer of words, sentences, and punctuation.

import nlp from 'compromise/one'
let doc = nlp("Wayne's World, party time")
let data = doc.json()
/* [{
  normal:"wayne's world party time",
    terms:[{ text: "Wayne's", normal: "wayne" },
      ...
      ]
  }]
*/

compromise/one splits your text up, wraps it in a handy API,

_{and does nothing else -}

/one is quick - most sentences take a 10th of a millisecond.

It can do ~1mb of text a second - or 10 wikipedia pages.

Infinite jest takes 3s.

compromise/two

A part-of-speech tagger, and grammar-interpreter.

import nlp from 'compromise/two'
let doc = nlp("Wayne's World, party time")
let str = doc.match('#Possessive #Noun').text()
// "Wayne's World"

_{this is more useful than people sometimes realize.}

Light grammar helps you write cleaner templates, and get closer to the information.

compromise has 83 tags, arranged in a handsome graph.

#FirstName → #Person → #ProperNoun → #Noun

you can see the grammar of each word by running doc.debug()

you can see the reasoning for each tag with nlp.verbose('tagger').

if you prefer Penn tags, you can derive them with:

let doc = nlp('welcome thrillho')
doc.compute('penn')
doc.json()

compromise/three

Phrase and sentence tooling.

import nlp from 'compromise/three'
let doc = nlp("Wayne's World, party time")
let str = doc.people().normalize().text()
// "wayne"

compromise/three is a set of tooling to zoom into and operate on parts of a text.

.numbers() grabs all the numbers in a document, for example - and extends it with new methods, like .subtract().

When you have a phrase, or group of words, you can see additional metadata about it with .json()

let doc = nlp('four out of five dentists')
console.log(doc.fractions().json())
/*[{
    text: 'four out of five',
    terms: [ [Object], [Object], [Object], [Object] ],
    fraction: { numerator: 4, denominator: 5, decimal: 0.8 }
  }
]*/

let doc = nlp('$4.09CAD')
doc.money().json()
/*[{
    text: '$4.09CAD',
    terms: [ [Object] ],
    number: { prefix: '$', num: 4.09, suffix: 'cad'}
  }
]*/

🤖 For AI agents & LLMs

Plain-text, copy-paste docs that render without JavaScript live in this repo:

• AGENTS.md — start here: mental model, rules, and gotchas
• docs/concepts.md — the document/View/Term model, mutability, build tiers
• docs/match-syntax.md — the .match() mini-language
• docs/tags.md — the complete, valid part-of-speech tagset
• docs/api.md — every method, signature, and description
• docs/recipes.md — solutions to common tasks
• llms-full.txt — all of the above in one fetchable file

API

Compromise/one

##### Output

• .text() - return the document as text
• .json() - return the document as data
• .debug() - pretty-print the interpreted document
• .out() - a named or custom output
• .html({}) - output custom html tags for matches
• .wrap({}) - produce custom output for document matches

• .found _[getter]_ - is this document empty?
• .docs _[getter]_ get term objects as json
• .length _[getter]_ - count the # of characters in the document (string length)
• .isView _[getter]_ - identify a compromise object
• .compute() - run a named analysis on the document
• .clone() - deep-copy the document, so that no references remain
• .termList() - return a flat list of all Term objects in match
• .cache({}) - freeze the current state of the document, for speed-purposes
• .uncache() - un-freezes the current state of the document, so it may be transformed
• .freeze({}) - prevent any tags from being removed, in these terms
• .unfreeze({}) - allow tags to change again, as default

• .all() - return the whole original document ('zoom out')
• .terms() - split-up results by each individual term
• .first(n) - use only the first result(s)
• .last(n) - use only the last result(s)
• .slice(n,n) - grab a subset of the results
• .eq(n) - use only the nth result
• .firstTerms() - get the first word in each match
• .lastTerms() - get the end word in each match
• .fullSentences() - get the whole sentence for each match
• .groups() - grab any named capture-groups from a match
• .wordCount() - count the # of terms in the document
• .confidence() - an average score for pos tag interpretations

_(match methods use the match-syntax.)_

• .match('') - return a new Doc, with this one as a parent
• .not('') - return all results except for this
• .matchOne('') - return only the first match
• .if('') - return each current phrase, only if it contains this match ('only')
• .ifNo('') - Filter-out any current phrases that have this match ('notIf')
• .has('') - Return a boolean if this match exists
• .before('') - return all terms before a match, in each phrase
• .after('') - return all terms after a match, in each phrase
• .union() - return combined matches without duplicates
• .intersection() - return only duplicate matches
• .complement() - get everything not in another match
• .settle() - remove overlaps from matches
• .growRight('') - add any matching terms immediately after each match
• .growLeft('') - add any matching terms immediately before each match
• .grow('') - add any matching terms before or after each match
• .sweep(net) - apply a series of match objects to the document
• .splitOn('') - return a Document with three parts for every match ('splitOn')
• .splitBefore('') - partition a phrase before each matching segment
• .splitAfter('') - partition a phrase after each matching segment
• .join() - merge any neighbouring terms in each match
• .joinIf(leftMatch, rightMatch) - merge any neighbouring terms under given conditions
• [.lookup([])](https://observablehq.com/@spencermountain/compromise-match) - quick find for an array of string matches
• .autoFill() - create type-ahead assumptions on the document

• .tag('') - Give all terms the given tag
• .tagSafe('') - Only apply tag to terms if it is consistent with current tags
• .unTag('') - Remove this term from the given terms
• .canBe('') - return only the terms that can be this tag

• .toLowerCase() - turn every letter of every term to lower-cse
• .toUpperCase() - turn every letter of every term to upper case
• .toTitleCase() - upper-case the first letter of each term
• .toCamelCase() - remove whitespace and title-case each term

• .pre('') - add this punctuation or whitespace before each match
• .post('') - add this punctuation or whitespace after each match
• .trim() - remove start and end whitespace
• .hyphenate() - connect words with hyphen, and remove whitespace
• .dehyphenate() - remove hyphens between words, and set whitespace
• .toQuotations() - add quotation marks around these matches
• .toParentheses() - add brackets around these matches

• .map(fn) - run each phrase through a function, and create a new document
• .forEach(fn) - run a function on each phrase, as an individual document
• .filter(fn) - return only the phrases that return true
• .find(fn) - return a document with only the first phrase that matches
• .some(fn) - return true or false if there is one matching phrase
• .random(fn) - sample a subset of the results

• .replace(match, replace) - search and replace match with new content
• .replaceWith(replace) - substitute-in new text
• .remove() - fully remove these terms from the document
• .insertBefore(str) - add these new terms to the front of each match (prepend)
• .insertAfter(str) - add these new terms to the end of each match (append)
• .concat() - add these new things to the end
• .swap(fromLemma, toLemma) - smart replace of root-words,using proper conjugation

• .sort('method') - re-arrange the order of the matches (in place)
• .reverse() - reverse the order of the matches, but not the words
• .unique() - remove any duplicate matches

_(these methods are on the main nlp object)_

• nlp.tokenize(str) - parse text without running POS-tagging
• nlp.lazy(str, match) - scan through a text with minimal analysis
• nlp.plugin({}) - mix in a compromise-plugin
• nlp.parseMatch(str) - pre-parse any match statements into json
• nlp.world() - grab or change library internals
• nlp.model() - grab all current linguistic data
• nlp.methods() - grab or change internal methods
• nlp.hooks() - see which compute methods run automatically
• nlp.verbose(mode) - log our decision-making for debugging
• nlp.version - current semver version of the library

• nlp.addWords(obj, isFrozen?) - add new words to the lexicon
• nlp.addTags(obj) - add new tags to the tagSet
• nlp.typeahead(arr) - add words to the auto-fill dictionary
• nlp.fromSpec(spec) - parse a .out('spec') output back into a document
• nlp.testSpec(spec, verbose?, throwError?) - test tags against a spec-formatted output
• nlp.buildTrie(arr) - compile a list of words into a fast lookup form
• nlp.buildNet(arr) - compile a list of matches into a fast match form

compromise/two:

##### Contractions

• .contractions() - things like "didn't"
• .contractions().expand() - things like "didn't"
• .contract() - things like "didn't"

compromise/three:

• .normalize({}) - clean-up the text in various ways

• .nouns() - return any subsequent terms tagged as a Noun

##### Verbs

• .verbs() - return any subsequent terms tagged as a Verb

##### Numbers

• .numbers() - grab all written and numeric values

• .money() - things like '$2.50'

• .fractions() - like '2/3rds' or 'one out of five'

• .percentages() - like '2.5%'

##### Sentences

• .sentences() - return a sentence class with additional methods

##### Adjectives

• .adjectives() - things like 'quick'

##### Misc selections

• .clauses() - split-up sentences into multi-term phrases
• .chunks() - split-up sentences noun-phrases and verb-phrases
• .hyphenated() - all terms connected with a hyphen or dash like 'wash-out'
• .phoneNumbers() - things like '(939) 555-0113'
• .addresses() - things like '23 Park Avenue'
• .hashTags() - things like '#nlp'
• .emails() - things like '[email protected]'
• .emoticons() - things like :)
• .emojis() - things like 💋
• .atMentions() - things like '@nlp_compromise'
• .urls() - things like 'compromise.cool'
• .pronouns() - things like 'he'
• .conjunctions() - things like 'but'
• .prepositions() - things like 'of'
• .abbreviations() - things like 'Mrs.'
• .people() - names like 'John F. Kennedy'

• .places() - like 'Paris, France'
• .organizations() - like 'Google, Inc'
• .topics() - people() + places() + organizations()
• .adverbs() - things like 'quickly'

• .acronyms() - things like 'FBI'

• .parentheses() - return anything inside (parentheses)

• .possessives() - things like "Spencer's"

• .quotations() - return any terms inside paired quotation marks

• .slashes() - return any terms grouped by slashes

.extend():

This library comes with a considerate, common-sense baseline for english grammar.

You're free to change, or lay-waste to any settings - which is the fun part actually.

the easiest part is just to suggest tags for any given words:

let myWords = {
  kermit: 'FirstName',
  fozzie: 'FirstName',
}
let doc = nlp(muppetText, myWords)

or make heavier changes with a compromise-plugin.

import nlp from 'compromise'
nlp.extend({
  // add new tags
  tags: {
    Character: {
      isA: 'Person',
      notA: 'Adjective',
    },
  },
  // add or change words in the lexicon
  words: {
    kermit: 'Character',
    gonzo: 'Character',
  },
  // change inflections
  irregulars: {
    get: {
      pastTense: 'gotten',
      gerund: 'gettin',
    },
  },
  // add new methods to compromise
  api: View => {
    View.prototype.kermitVoice = function () {
      this.sentences().prepend('well,')
      this.match('i [(am|was)]').prepend('um,')
      return this
    }
  },
})

Docs:

##### gentle introduction:

• #1) Input → output
• #2) Match & transform
• #3) Making a chat-bot

##### Documentation:

| Concepts | API | Plugins | | ------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------: | -------------------------------------------------------------------------------------: | | Accuracy | Accessors | Adjectives | | Caching | Constructor-methods | Dates | | Case | Contractions | Export | | Filesize | Insert | Hash | | Internals | Json | Html | | Justification | Character Offsets | Keypress | | Lexicon | Loops | Ngrams | | Match-syntax | Match | Numbers | | Performance | Nouns | Paragraphs | | Plugins | Output | Scan | | Projects | Selections | Sentences | | Tagger | Sorting | Syllables | | Tags | Split | Pronounce | | Tokenization | Text | [Strict]

... (README truncated for length)