oborchers/Fast_Sentence_Embeddings: Compute Sentence Embeddings Fast!

Fast Sentence Embeddings ==================================

Status ------------

fse 2.0 is in development on master and not released yet; this README describes it, and its API may still change before 2.0.0. The current release is fse 1.1.0 on PyPI, in maintenance mode on the branch 1.x (bug and compatibility fixes only), with its own README and notebooks. 2.0 changes the API and the file format of saved models. Feature requests are closed until then; bug reports are welcome.

If you start a new project, also look at actively maintained libraries such as sentence-transformers or model2vec.

About ------------

fse 2.0 encodes text with finished static embedding models: a model (fse.Encoder) is one .safetensors file that holds a table of word rows, its tokenizer and whether every row is normalized. A sentence becomes the mean of the rows of its words, computed in parallel by a native kernel, bit for bit the same on every CPU. fse 2.0 trains nothing: weights, removed components or projections of a model (such as SIF or uSIF) are folded into its rows before it is saved. Vectors go into memory, or into a store on disk that fse.VectorStore searches.

Installation ------------

2.0 is not on PyPI yet. From a checkout of master, with uv and a C compiler:

make install

fse.gensim (converting gensim word vectors) needs the extra fse[gensim].

On Apple silicon with macOS 14 or newer, the extra fse[mlx] adds the backend "mlx", which averages the word vectors on the GPU within a small tolerance of the CPU's rows (usually with their bits): encoder.encode(texts, backend="mlx"). For tiny inputs it is slower than the default "cpu".

Usage ------------

Load a finished model and encode:

import fse
encoder = fse.Encoder.load("model.safetensors")
rows = encoder.encode(["the cat sleeps", "a dog runs"])  # one float32 row per text

Make a model from word vectors (every word weighs 1) and save it:

import fse
from fse.gensim import from_gensim
from gensim.models import KeyedVectors
wv = from_gensim(KeyedVectors.load("glove-wiki-gigaword-300.model"))
encoder = fse.Encoder.from_word_vectors(wv)  # tokenizer fse_words@1, rows normalized
encoder.save("glove.safetensors")  # one file with the table

Which call to use when:

| You want to | Call | |---|---| | load a model | encoder = fse.Encoder.load("model.safetensors") (mmap=True maps it read-only) | | make a model of word vectors | fse.Encoder.from_word_vectors(wv), or fse.Encoder.from_arrays(vectors, words=words) | | keep the mean as it is | normalize=False | | encode texts in a list | rows = encoder.encode(texts) | | encode token lists you made yourself | a model with tokenizer="external" and lists of lists of str | | encode a text file, one text per line | encoder.encode(Path("texts.txt")) | | encode a stream (a generator, a queue) | encoder.encode(iterator): it is read once | | encode a stream batch by batch and confirm what is safe | for batch in encoder.encode_batches(polls, sink=sink), confirming up to batch.durable_rows | | write the vectors to disk | with fse.ShardSink("store", encoder, dtype="int8") as sink: encoder.encode(texts, sink=sink) | | search them | with fse.VectorStore.open("store") as store: result = store.search(encoder.encode(["query"]), k=10) |

encode returns float32 rows by default; dtype="float16" gives float16 rows and dtype="int8" a fse.QuantizedRows (int8 codes and one scale per row).
store.search returns a fse.SearchResult with the ids (positions in the store) and the cosine scores, best first. Its default mode="auto" takes the exact arithmetic of fse for up to 8 queries at once and BLAS above, whose bits depend on how many queries are asked together; mode="exact" gives the same results for every batch.
Texts are tokenized by fse_words@1 by default: lowercase, split at whitespace and punctuation, contractions expanded (ARCHITECTURE.md, "Tokenizers"). Subword tokenizers (WordPiece, Unigram), which models such as POTION use, follow in a later PR; until then such models run with tokenizer="external" and tokens you make.
FastText word vectors keep their ngram rows: an unknown word adds the rows of its first max_ngrams ngrams (default 40) with the weight oov_weight (default 1).

Classify and score text

fse runs linear heads trained elsewhere; it trains none. Train in scikit-learn on fse's vectors, then import the head:

import fse
from sklearn.linear_model import LogisticRegression
encoder = fse.Encoder.load("model.safetensors")  # or from_word_vectors(wv, normalize=False)
x = encoder.encode(train_texts)
head = LogisticRegression(max_iter=1000).fit(x, train_labels)
clf = fse.Classifier.from_sklearn(encoder, head)  # reads coef_, intercept_, classes_
clf.predict(texts)  # labels, as head.predict(encoder.encode(texts)) gives them
clf.predict_proba(texts)  # float32 (n, classes)
for batch in clf.predict_batches(polls, output="proba"):  # a stream, batch by batch
    handle(batch.outputs)
clf.save("classifier.safetensors")  # encoder and head
clf.save("filter.safetensors", standalone=True)  # the folded table and the bias only

Train on raw means (an encoder with normalize=False) when you only need the predictions: such a head folds into the table (every word row times A), so predicting reads k values per word instead of d and is faster than encode. A head trained on normalized vectors runs on the full mean; normalized= says which one a head is, by default the encoder's setting.
fse.Regressor does the same for regression heads (predict); from_arrays(encoder, weight, bias, labels=...) takes a head of your own.
predict and predict_proba also take a fse.VectorStore of the encoder's rows (store mode: the texts are not encoded again), and predict_batches(..., return_vectors=True) gives the vectors with the predictions from one pooling.
The predictions are bit-identical across CPU backends, worker counts and runs. The folded head and the vectors agree with scikit-learn within float32 rounding: on Amazon Reviews Polarity (GloVe 300, LogisticRegression on 400,000 reviews, 400,000 test reviews) fse and scikit-learn give the same label for all but one review on raw means (its logit 7.4e-7 from the decision boundary) and for every review on normalized vectors, an accuracy of 0.8299 and 0.8284, and probabilities within 1.2e-6.

What 2.0 no longer has, against fse 1.1.0 and the earlier development versions of 2.0:

training: Average, SIF and uSIF with train or fit, and any learning from a corpus. SIF and uSIF tables are made offline; a notebook with the 1.1.0 recipe (word weights and the removal of principal components, folded into the table) follows before the release.
stored recipes and learned components: a model holds its finished table, not the recipe, the weights or the components it was made with.
word frequencies: word vectors hold no counts, and from_gensim takes none.
shared external tables: every model holds its own table, so two models of the same word vectors hold two copies.
Corpus: encode takes lists, paths and streams directly.
Files written by a development version of 2.0 before this change are refused (fse.UnsupportedArtifactError); rebuild them.

ARCHITECTURE.md describes the design and the numerical contract, CHANGELOG.md every change against 1.1.0.

Results ------------

These are results of fse 1.1.0, not of 2.0 models; the offline SIF and uSIF recipe will be checked against them. Pearson correlation (x100) on the test set of the STS Benchmark, computed with the procedure of notebooks/STS-Benchmarks.ipynb on the branch 1.x (lang_freq="en") with fse 1.1.0 on Python 3.13, numpy 2.5, scipy 1.18, gensim 4.4, wordfreq 3.1 and nltk 3.10. SIF and uSIF compute their principal components exactly since 1.1.0; against the randomized SVD of scikit-learn 1.9 used before, in the same environment, SIF-10 scores move by at most 0.04 points and uSIF scores not at all. Compared with the table published for fse 1.0, uSIF scores rise by 0.2 to 1.4 points. Most of that comes from a fix in 1.1.0 to how uSIF weights the removed principal components, worth 0.2 to 1.3 points when both formulas run in this environment (see CHANGELOG.md). Newer dependency versions shift SIF scores by up to 0.3 points and CBOW scores by up to 0.04 points.

Literature -------------

Iyyer M, Manjunatha V, Boyd-Graber J, Daumé III H (2015) Deep Unordered

Composition Rivals Syntactic Methods for Text Classification. Proc. 53rd Annu. Meet. Assoc. Comput. Linguist. 7th Int. Jt. Conf. Nat. Lang. Process., 1681-1691.

Arora S, Liang Y, Ma T (2017) A Simple but Tough-to-Beat Baseline for Sentence

Embeddings. Int. Conf. Learn. Represent. (Toulon, France), 1-16.

Ethayarajh K (2018) Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline.

Proceedings of the 3rd Workshop on Representation Learning for NLP. (Toulon, France), 91-100.

Eneko Agirre, Daniel Cer, Mona Diab, Iñigo Lopez-Gazpio, Lucia Specia. Semeval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation. Proceedings of SemEval 2017.

Changelog -------------

See CHANGELOG.md.

Development -------------

The project uses uv and a Makefile:

make install # create the environment and build the extension make check # format check, lint, type check and tests

See CONTRIBUTING.md for details and SECURITY.md for reporting vulnerabilities.

Author -------------

Built by Dr. Oliver Borchers. For questions that do not belong in a GitHub issue, use the contact page.

Citation -------------

If you found this software useful, please cite it in your publication.

@misc{Borchers2019, author = {Borchers, Oliver}, title = {Fast sentence embeddings}, year = {2019}, publisher = {GitHub}, journal = {GitHub Repository}, howpublished = {\url{https://github.com/oborchers/Fast_Sentence_Embeddings}}, }

oborchers/Fast_Sentence_Embeddings: Compute Sentence Embeddings Fast!

Classify and score text

Reading Mode

Related Stories

CVE-2026-98432: Linux TCP Use-After-Free Risk and Patch Validation

Show HN: Lasm – Write formally verified webapps in Lean for Node/serverless/edge

Pico CSS is no longer maintained

Herdr has released native app

Cormac McCarthy's Library

Related Stories