Profile
Back to NewsBack
GitHub Trending 35 min
Reader Mode
liberation-data/drivine: Best and fastest graph database client (Neo4j, AgensGraph, Apache AGE, FalkorDB) for Node.js & TypeScript.

liberation-data/drivine: Best and fastest graph database client (Neo4j, AgensGraph, Apache AGE, FalkorDB) for Node.js & TypeScript.

4 hours ago

Drivine: graph database client for TypeScript, Python and the JVM

drivine

Docs: drivine.org, for TypeScript, Python, and Kotlin & Java.

Drivine is a graph database client for Node.js and TypeScript, with a NestJS module, and no need for one: see Without Nest. It supports Neo4j, Memgraph, FalkorDB, Oracle (SQL/PGQ) and Postgres as graph stores, plus Amazon Neptune through the low-level API. It gives you two ways to work with a graph:

  1. PersistenceManager: a low-level API where you write the queries yourself. This is the classic Drivine approach.
  2. Object manager (StatelessGraphObjectManager): a high-level API that maps decorated classes, with a typed
query DSL whose references come from the classes themselves.
On Python or the JVM? drivine-python (pip install drivine)
is Drivine's Python twin, async first with a sync API. drivine4j
(org.drivine:drivine4j) is its Java and Kotlin twin, the graph client for Embabel, with
Spring Boot support and a code-generated DSL. The three share a storage format: a store written from any of them is
read, stamped and checked alike by the others.

Its design goals:

  • Support multiple graph databases, simultaneously if you wish.
  • Scale to hundreds and thousands of transactions per second, without compromising architectural integrity.

Contents

- With Nest - Without Nest - Key Concepts - Loading Data - Query DSL - Grouped Reads - Saving Data - @NodeStamp - Deleting Data - Polymorphism - Mapping Decorators - Relationships of a Runtime Type

Philosophy

Composition Over Inheritance

A typical ORM defines one reusable object model, and generates statements to move data between that model and the database. That addresses the so-called impedance mismatch, at a cost:

  • Generated queries work well for simple cases, but can get out of hand and perform badly when things get complex.
Debugging generated statements can be painful.
  • One model for many use cases is a big ask. The original CRUD cases work well, but in harder cases the model gets in
the way more than it helps.

Those trade-offs may be acceptable for a relational database, but with a graph database the mismatch hardly exists.

Just as we favour composition over inheritance in software, we prefer composition when mapping the results of complex queries. A person plays many roles: sometimes we help them have a great holiday, sometimes they manage a team, sometimes they're a person of interest. With Drivine you compose a view for each use case:

@GraphView()
export class HolidayingPerson {
    @Root() person: Person;

@GraphRelationship({ type: 'BOOKED_HOLIDAY', target: () => Holiday }) holidays: Holiday[]; }

Behind the scenes, Drivine writes one efficient statement:

MATCH (person:Person)
WITH person, [(person)-[:BOOKED_HOLIDAY]->(holiday:Holiday) | holiday {.*}] AS holidays
RETURN { person: properties(person), holidays: holidays }

Composition lets you mix and match as needed.

Installation

npm install @liberation-data/drivine

Drivine needs reflect-metadata and neo4j-driver as peer dependencies, and NestJS (6 to 11) only for its module. The FalkorDB, Oracle (oracledb, thin mode, no Instant Client needed) and Postgres drivers are bundled.

The object manager reads decorators and their type metadata, so enable both in tsconfig.json:

{
    "compilerOptions": {
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true
    }
}

Quick Start

1. Configuration

Connections come from the environment, or a .env file that Drivine loads. The default database is configured with DATABASE_, and a named one with _DATABASE_:

DATABASE_TYPE=NEO4J          # NEO4J, MEMGRAPH, FALKORDB, ORACLE, NEPTUNE or POSTGRES
DATABASE_HOST=localhost
DATABASE_PORT=7687
DATABASE_USER=neo4j
DATABASE_PASSWORD=password
DATABASE_NAME=neo4j          # optional on the graph engines; on Oracle, the service, e.g. FREEPDB1
DATABASE_POOL_MAX=40
DATABASE_CONNECTION_TIMEOUT=5000

Use them through Nest's module, or without Nest. Only the setup differs: the decorators, the DSL, @Transactional and the managers are the same.

2. With Nest

Register the connections with the module, from @liberation-data/drivine/nest:

import { DrivineModule } from '@liberation-data/drivine/nest';

@Module({ imports: [ DrivineModule.withOptions({ connectionProviders: [DatabaseRegistry.buildOrResolveFromEnv()] }) ], providers: [RouteRepository, PersonService] }) export class AppModule {}

Then inject a persistence manager, an object manager or a statement. The Inject… decorators come from @liberation-data/drivine/nest too; everything else from @liberation-data/drivine:

@Injectable()
export class RouteRepository {
    constructor(
        @InjectPersistenceManager() readonly persistenceManager: PersistenceManager,
        @InjectCypher(__dirname, 'routesBetween') readonly routesBetween: CypherStatement
    ) {}

@Transactional() // Propagation.REQUIRED by default: join the current transaction, or start one. async findFastestBetween(start: string, destination: string): Promise<Route> { return this.persistenceManager.getOne( new QuerySpecification<Route>() .withStatement(this.routesBetween) .bind([start, destination]) .limit(1) .transform(Route) ); } }

Or, with the object manager, declare what you want and let Drivine write the statement:

@Injectable()
export class PersonService {
    constructor(@InjectGraphObjectManager() readonly gom: StatelessGraphObjectManager) {}

async leads(): Promise<PersonCareer[]> { return this.gom.loadAll(PersonCareer, (q) => ({ where: q.person.bio.containsIgnoreCase('lead') })); } }

The module closes the connections when the application stops.

3. Without Nest

In Express, Fastify, a job or a CLI, make the factories yourself, once, and hand the managers to your classes:

import {
    cypherStatement,
    DatabaseRegistry,
    GraphObjectManagerFactory,
    inDrivineContext,
    PersistenceManager,
    PersistenceManagerFactory,
    QuerySpecification,
    StatelessGraphObjectManager,
    Transactional
} from '@liberation-data/drivine';
import { readFileSync } from 'fs';

DatabaseRegistry.buildOrResolveFromEnv(); // and buildOrResolveFromEnv('ANALYTICS'), … const persistenceManagers = new PersistenceManagerFactory(); const graphObjectManagers = new GraphObjectManagerFactory(persistenceManagers);

export class RouteRepository { private readonly routesBetween = cypherStatement(readFileSync(${__dirname}/routesBetween.cypher, 'utf8'));

constructor(readonly persistenceManager: PersistenceManager = persistenceManagers.get()) {}

@Transactional() // the same as in Nest: join the current transaction, or start one async findFastestBetween(start: string, destination: string): Promise<Route> { return this.persistenceManager.getOne( new QuerySpecification<Route>() .withStatement(this.routesBetween) .bind([start, destination]) .limit(1) .transform(Route) ); } }

export class PersonService { constructor(readonly gom: StatelessGraphObjectManager = graphObjectManagers.get()) {}

async leads(): Promise<PersonCareer[]> { return this.gom.loadAll(PersonCareer, (q) => ({ where: q.person.bio.containsIgnoreCase('lead') })); }

@Transactional() async hire(person: Person): Promise<void> { await this.gom.save(person); } }

@Transactional needs no setup. Drivine keeps the transaction in Node's AsyncLocalStorage, which follows every await. A marked method starts a transaction when none is running, and every marked method it calls, in any class, joins it. Nothing passes a transaction or a context around:

class AuditLog {
    constructor(readonly gom: StatelessGraphObjectManager = graphObjectManagers.get('AUDIT')) {}

@Transactional() async record(event: AuditEvent): Promise<void> { await this.gom.save(event); } }

class Onboarding { constructor(readonly people = new PersonService(), readonly audit = new AuditLog()) {}

@Transactional() async onboard(person: Person): Promise<void> { await this.people.hire(person); // joins onboard's transaction await this.audit.record(AuditEvent.hired(person)); // so does this, on another database } // both commit here; a throw anywhere in it rolls both back }

To run a block in one transaction without a decorator:

await inDrivineContext()
    .withTransaction()
    .run(async () => { / … / });

Close the connections when the application stops:

await Promise.all(DatabaseRegistry.getInstance().providers.map(async (it) => it.end()));

sample/lambda is a runnable example: an AWS Lambda handler, export const handler, with the factories made once per cold start, a repository, and a @Transactional service that rolls a hire back when it takes an organization past its headcount. npm run start-lambda-sample serves it on your machine, on PORT (3000) and over the database DATABASE_* configures:

curl -X POST localhost:3000/hires -d '{"name": "Ada", "bio": "Team Lead", "organization": "Acme", "role": "CTO"}'
curl localhost:3000/leads

Return One Map

When you write a statement yourself, return a single map or a scalar, not several columns. That maps cleanly with .transform() and avoids trouble with nulls:

-- Wrong: several columns, awkward to map
RETURN a.name, a.age, b.title

-- Right: one map RETURN { name: a.name, age: a.age, title: b.title } AS result

-- Right: a node's properties, or a scalar RETURN properties(p) RETURN count(p) AS total

Object Manager

StatelessGraphObjectManager maps decorated classes to the graph. It writes the statements itself, and gives you a typed DSL for filtering and ordering.

It keeps no state between calls. A load returns what the store holds. What a save writes depends only on the object and the options you pass.

Inject one for each database:

constructor(
    @InjectGraphObjectManager() readonly gom: StatelessGraphObjectManager,              // the default database
    @InjectGraphObjectManager('ANALYTICS') readonly analytics: StatelessGraphObjectManager
) {}

Or take one from GraphObjectManagerFactory, which the module provides: factory.get('ANALYTICS'). Without Nest, from the factory you made: graphObjectManagers.get('ANALYTICS').

Key Concepts

1. NodeFragment: a Node

A @NodeFragment maps one node. Every property of the object is stored, unless it is marked @GraphTransient():

@NodeFragment({ labels: ['Person'] })
export class Person {
    @NodeId() uuid: string;
    name: string;
    bio?: string;
}

A fragment names at least one label. A subclass carries its own labels as well as its superclass's: see Polymorphism.

Defaults for missing properties. A property the node doesn't have keeps the class's initializer. Two decorators handle a stored null:

@NodeFragment({ labels: ['User'] })
export class UserNode {
    @NodeId() id: string;
    @Default() status = 'active';        // a stored null loads as 'active'
    @EmptyWhenAbsent() tags: string[];   // absent or null loads as []; an object loads as {}
}

2. RelationshipFragment: a Relationship's Properties

A @RelationshipFragment holds the properties of a relationship, and the node it leads to:

@RelationshipFragment()
export class WorkHistory {
    startDate: string;                  // a property of the relationship
    role: string;                       // and another
    @TargetNode() target: Organization; // the node it leads to
}

Use it for employment history, transactions, audit trails, or any relationship with data of its own.

3. GraphView: Composing Fragments

A @GraphView composes fragments and relationships into one result. @Root marks where the query starts:

@GraphView()
export class PersonCareer {
    @Root() person: Person;

@GraphRelationship({ type: 'WORKS_FOR', target: () => WorkHistory }) employmentHistory: WorkHistory[] = []; }

@GraphRelationship takes:

| Option | Meaning | |---|---| | type | The relationship type. | | direction | Direction.OUTGOING (the default), INCOMING or UNDIRECTED. | | target | The class of the related item: a fragment, a relationship fragment or a view, as a function so that classes may refer to each other in either order. It is required for a list, since TypeScript records only Array as a list's type. For a single item it defaults to the property's type. | | required | Only roots that have the relationship are loaded. TypeScript erases ?:, so a view says this rather than the type: see Required Relationships. | | maxDepth | For a recursive relationship, how deep to expand. Defaults to 1. |

When a view refers to a class declared later in the same module, type the property with Related, so the decorator metadata doesn't read the class before it exists: @GraphRelationship({ type: 'TO_B', target: () => B }) b?: Related.

4. Recursive Relationships: Hierarchies and Ontologies

A view whose relationship targets its own class is expanded to maxDepth levels:

@NodeFragment({ labels: ['Location'] })
export class Location {
    @NodeId() uuid: string;
    name: string;
    type: string;
}

@GraphView() export class LocationHierarchy { @Root() location: Location;

@GraphRelationship({ type: 'HAS_LOCATION', target: () => LocationHierarchy, maxDepth: 3 }) subLocations: LocationHierarchy[]; }

At the last level a list is [] and a single item is undefined. To walk upwards, make another view with direction: Direction.INCOMING. A query can override the depth:

await gom.loadAll(LocationHierarchy, (q) => ({
    where: q.location.type.eq('continent'),
    depth: { subLocations: 5 }
}));

Cycles through other views (A → B → A) are expanded to maxDepth as well. A view nested in itself is saved, each level one relationship from the one above. A list of fragments read over several hops (maxDepth above 1 on a list of a fragment) is loaded and never written.

5. Path Traversal: Skipping Intermediate Nodes

@GraphRelationship is one hop. @GraphPath follows several and maps only the node at the end:

@GraphView()
export class ActorDirectors {
    @Root() actor: Actor;

@GraphPath([hop('ACTED_IN', Direction.OUTGOING, 'Movie'), hop('DIRECTED_BY')], { target: () => Director }) directors: Director[]; }

Each node at the end is returned once: an actor who made two movies with one director gets that director once. A hop's label constrains the node it reaches. required: true keeps only the roots the path leads somewhere from. A path is read-only.

One hop may name an order: a property of the node it reaches, lowest first unless ascending is false. A list then holds its targets in the order of the first node of that hop that leads to each, and a single field holds the target the first such node leads to:

@GraphView()
export class SessionStarter {
    @Root() session: Session;

// who wrote the earliest message @GraphPath([hop('HAS_MESSAGE', Direction.OUTGOING, 'Message', 'sentAt'), hop('AUTHORED_BY')], { target: () => Person }) startedBy?: Person; }

A node of the ordered hop without the property comes last, whichever way the path is read. Targets whose first nodes hold equal values come in no set order. The property should hold values of one type: every backend reads the order the same way where it does, a text of digits such as '0100' is ordered as text, and an empty text is a text, the lowest one. Where it holds texts on some nodes and numbers on others, every backend reads every text before every number lowest first, and after them highest first, as Neo4j's ORDER BY does. A group key's path names no order: pathKey refuses one.

6. Aggregates: Counting and Summarising Without Loading

@Count and @Aggregate compute a number for each root in the query, so you don't load a list just to size it:

@GraphView()
export class ActorStats {
    @Root() actor: Actor;
    @Count('ACTED_IN') movieCount: number;
    @Aggregate({ function: AggregateFunction.AVG, type: 'RATED', property: 'score' }) avgRating?: number;
    @Aggregate({ function: AggregateFunction.SUM, type: 'RATED', property: 'score' }) totalRating: number;
}

SUM and COUNT over nothing are 0; AVG, MIN and MAX over nothing are undefined. Aggregates follow one hop and belong to one root. To group nodes by a key and rank the groups, use a grouped read.

Loading Data

const everyone = await gom.loadAll(PersonCareer);
const one = await gom.load(PersonCareer, uuid);                          // undefined when there is none
const open = await gom.loadAll(Issue, (q) => ({ where: q.state.eq('open') }));
const raw = await gom.loadAll(Issue, 'n.state = $state', { state: 'open' }); // a predicate of your own, over n

Count

count counts exactly what loadAll would return for the same type and filter:

await gom.count(Issue);
await gom.count(Issue, (q) => ({ where: q.state.eq('open') }));
await gom.count(RaisedAndAssignedIssue);   // only the issues that meet the view's required relationships

A fragment is counted by its labels. A view counts only the roots that meet its required relationships and paths, so the number equals loadAll(...).length, not a count of nodes.

Vector Search (loadNearest)

loadNearest runs an approximate nearest-neighbour search and returns each hit with a similarity score. It works on a view, by its root's embedding, and on a fragment:

@NodeFragment({ labels: ['Proposition'] })
export class PropositionNode {
    @NodeId() id: string;
    text: string;
    @VectorIndex({ similarity: 'COSINE' }) embedding?: number[];
}

const hits: Scored<PropositionView>[] = await gom.loadNearest(PropositionView, queryEmbedding, { topK: 20 }); hits.forEach((it) => console.log(it.score, it.value.proposition.text));

The searched embedding is the fragment's @VectorIndex field: inferred when there is one, and named with property when there are several. The score is a similarity, higher meaning more similar, on every engine, so threshold means the same thing everywhere.

| Option | Meaning | |---|---| | topK | How many to return. Without searchK, also how many the index is asked for. | | searchK | How many the index is asked for, at least topK. topK then trims after the filter. | | threshold | The least score a hit may have. | | where | A condition applied after the search: references, or a predicate over n with bindings. | | partitionLabel | Search the index of another label over the same property: one index per corpus or tenant. | | property | Which @VectorIndex field, when the fragment has several. | | depth | maxDepth overrides for a view. |

topK is the index's k, not a promised count. A view's required relationships, threshold and where filter after the search, so a search can return fewer than topK.

k is also the search's beam. On Lucene-backed engines the result queue is the HNSW candidate queue, so a small k can miss a vector that really is nearest. searchK widens the search without widening the result:

// search with a beam of 200, and return the best 40 that pass the filter
await gom.loadNearest(PropositionView, queryEmbedding, {
    topK: 40,
    searchK: 200,
    where: (q) => [q.proposition.contextId.eq(ctx), q.mentions.any((m) => m.resolvedId.eq(entityId))]
});

A filtered search dilutes: at 22% selectivity, 40 asked for returns about 9, the nearest overall that happen to pass rather than the nearest of those that pass. searchK mitigates that. A per-partition index avoids it: with one vector index for each partition label, partitionLabel searches inside the partition, which is filtered by construction.

Full-Text Search (loadMatching)

loadMatching is the full-text mirror of loadNearest. The relevance is scaled to [0, 1], the best hit being 1:

@NodeFragment({ labels: ['Chunk'] })
export class ChunkNode {
    @NodeId() id: string;
    @FullTextIndex() text: string;
}

await gom.loadMatching(ChunkNode, 'graph databases', { topK: 20 }); await gom.loadMatching(ChunkView, 'graph databases', { topK: 20, threshold: 0.5, where: (q) => q.chunk.sectionId.eq('s1') });

The index is the fragment's @FullTextIndex: on a property for one field, or on the class, @FullTextIndex({ properties: ['title', 'body'] }), for several. Name one with property when the fragment declares more than one. The text is the engine's full-text language (Lucene on Neo4j): escape user input yourself.

Query DSL

A query is a function of the class's references, q, which mirror its fields. The compiler checks each one, so a query over a field the class doesn't have, or a comparison with a value of the wrong type, doesn't compile:

const leads = await gom.loadAll(PersonCareer, (q) => ({
    where: q.person.bio.contains('Lead'),
    orderBy: q.person.name.asc(),
    limit: 20
}));

A query may hold where, orderBy, limit, skip, seek and depth.

Conditions

// AND: a list of conditions
where: [q.person.name.eq('Alice'), q.person.bio.isNotNull()]

// OR, AND and NOT where: anyOf(q.person.name.eq('Alice'), q.person.name.eq('Bob')) where: [q.issue.state.eq('open'), anyOf(q.issue.id.eq(123), q.assignedTo.name.eq('Alice'))] where: not(q.issue.locked.eq(true))

// a relationship's target, or a nested view's root and relationships where: q.assignedTo.name.eq('Alice') where: q.raisedBy.worksFor.name.startsWith('Ac')

A condition on a relationship holds when at least one related node meets it. Each condition is its own existence check, so two conditions need not hold of the same related node. When they must, use any:

// propositions with at least one mention that resolves to this entity
where: q.mentions.any((m) => m.resolvedId.eq(entityId))

// one mention that is both the subject and resolves to this entity where: q.mentions.any((m) => [m.role.eq('SUBJECT'), m.resolvedId.eq(entityId)])

// propositions with no such mention, including those with no mentions at all where: q.mentions.none((m) => m.resolvedId.eq(entityId))

Operators

| Kind | Operators | |---|---| | Comparison | eq, neq, gt, gte, lt, lte, in, notIn | | Text | contains, startsWith, endsWith, matches (a regular expression; not on FalkorDB), containsIgnoreCase, eqIgnoreCase | | Null | isNull(), isNotNull() | | List property | hasItem(value): the value is in a list-valued property, the mirror of in | | Logic | anyOf(...), allOf(...), not(...), a list for AND | | Labels | instanceOf(Type): the node carries all the labels of a fragment class. hasAnyLabel('A', 'B'): it carries any of them | | Order | asc(), desc(), then nullsFirst() or nullsLast() |

gt, gte, lt and lte are offered on numbers, texts and dates, and the text operators on texts alone.

Querying Fragments

The DSL isn't only for views. Every fragment has references to its own fields:

await gom.loadAll(ChunkNode, (q) => ({ where: q.sectionId.eq('sec-1') }));
await gom.count(ChunkNode, (q) => ({ where: q.sectionId.eq('sec-1') }));
await gom.loadAll(WebUser, (q) => ({ where: q.instanceOf(AnonymousWebUser) }));

A reference uses the field's name. Drivine maps it to the stored name, whether @GraphProperty renamed it or it is a @NodeStamp field, and an entry of a @PropertyBag is reached through the bag: q.metadata.source.eq('wiki').

Runtime Keys

When the property isn't known until run time, such as an arbitrary bag key or a filter a caller names, give the stored name. Values are still bound as parameters:

where: q.property('metadata.source').eq('wiki')
where: q.predicate('metadata.tags', 'HAS_ITEM', 'kotlin')

A name that isn't a plain identifier is quoted for you, and can't break out of its quotes.

Ordering

orderBy: [q.person.name.asc(), q.person.uuid.asc()]
orderBy: q.proposition.confidence.desc().nullsLast()

Left to itself, the database decides where rows with no value go. Cypher ranks null above every value, so a descending order starts with them. nullsFirst() and nullsLast() say otherwise. No index supplies that order, so leave it off a property that is never null.

An order by an expression. A ranking computed from several properties is an expression over the root's properties, built from plus, minus, times, div, negate, exp, floor, coalesce, epochSeconds and caseOf. Every value you give is bound. The ranking runs in the database with its limit, and later terms break its ties:

orderBy: [
    caseOf([above(q.validFrom, asOf).or(atMost(q.validTo, asOf)).then(0)],
           times(q.confidence, exp(negate(times(q.decay, floor(div(minus(epochSeconds(asOf), q.lastAccessedSeconds), 86400)))))))
        .desc(),
    q.id.asc()
]

An expression reads only the root's properties, and is refused with seek.

Ordering a relationship's list. An order on a relationship's property sorts that list in each object, not the roots: orderBy: [q.issue.id.desc(), q.assignedTo.name.asc()] orders the issues by id, and each issue's assignees by name. A nested view's list sorts the same way: q.raisedBy.worksFor.name.desc(). This works the same on every engine and needs no plugin. A recursive list can't be ordered by the query: use @SortedBy, which orders every level.

@SortedBy sorts a list relationship after it is loaded, on every load of the view:

@GraphRelationship({ type: 'HAS_CONTRIBUTOR', target: () => Contributor })
@SortedBy('name')                      // ascending; @SortedBy('name', false) for descending
contributors: Contributor[];

@SortedBy('contributor.name') // a path into a related view or relationship fragment

Nulls go last.

Limit and Pagination

limit and skip are applied in the database, after the order:

await gom.loadAll(PropositionView, (q) => ({ orderBy: q.proposition.created.desc(), limit: 20 }));
await gom.loadAll(PropositionView, (q) => ({ orderBy: q.proposition.created.desc(), skip: 40, limit: 20 }));

For a view, limit bounds the roots: each view still has all its relationships. Pair it with orderBy, or the subset is arbitrary. count ignores limit and skip.

For large or changing results, page by keyset with seek. Its values follow the order's terms, in the same order. Drivine works out each comparison from the direction. End the order with a unique key, so that ties can't be skipped or repeated:

const page = await gom.loadAll(SessionView, (q) => ({
    orderBy: [q.session.lastActivityAt.desc(), q.session.sessionId.desc()],
    seek: cursor ? [q.session.lastActivityAt.after(cursor.lastActivityAt), q.session.sessionId.after(cursor.sessionId)] : undefined,
    limit: pageSize + 1   // one more than a page tells you whether there is a next one
}));
  • The ordered properties must not be null. A row whose key is null meets no comparison, so it drops out of every
page after the first.
  • seek is refused without a matching order, with a null cursor value, with skip, with an order that places its
nulls, and with an expression.
  • Index the cursor. A keyset is cheap only where the engine can seek into an index and stop. The index mirrors the
cursor: a range index over exactly the ordered properties, in order, such as @RangeIndex({ properties: ['lastActivityAt', 'sessionId'] }) on the class. Profiled on Neo4j with 200,000 nodes, a page from the middle took 27 database accesses with that index, 200,020 with an index on the first key alone, and 500,010 with none.
  • This is about Neo4j. Memgraph and FalkorDB sort between the scan and the limit whatever is indexed, so a keyset costs
what is left rather than a page. seek is still worth having there, for pages that don't shift under concurrent writes.

Drivine tells you when the index is missing. An ordered query is checked against the database's indexes, and one that nothing mirrors is reported, once for each label and set of properties. Ordering without an index is correct, only unindexed, so the default is to warn. Turn it up in development or CI:

DrivineModule.withOptions({
    connectionProviders: [DatabaseRegistry.buildOrResolveFromEnv()],
    query: { indexAdvice: 'FAIL' }   // 'WARN' (the default) or 'OFF'
});

That applies to every manager the module makes. Without Nest, give it to the factory: new GraphObjectManagerFactory(persistenceManagers, { indexAdvice: 'FAIL' }). To change one manager: gom.indexAdvice = 'FAIL'. A query pinned to one root, by an equality on the id or on a uniquely constrained property, is never advised on. The index list is read once and kept; after making indexes as the application runs, call gom.indexAdvisor.refresh().

Grouped Reads (groups)

gom.groups groups a fragment's nodes by a key and gives numbers about each group: a count, a sum, a ranking. It is one statement on every engine, Oracle included, and nothing is loaded to be tallied in the application.

Each part of a grouped read is a clause of the query you would write by hand:

| Here | Cypher | SQL | | --- | --- | --- | | groups.load(T, ...) | MATCH (n:T) | FROM t | | where | WHERE before the WITH | WHERE | | keys: { page: key(q.page) } | WITH n.page AS page, … | GROUP BY page | | keys: { user: pathKey(User, [...]) } | MATCH (n)-[…]->(user:User), then WITH user, … | a join, then GROUP BY its id | | aggregates: { votes: count() } | count(n) AS votes | COUNT(*) AS votes | | aggregates: { likes: count(q.helpful.eq(true)) } | sum(CASE WHEN … THEN 1 ELSE 0 END) AS likes | the same | | having | WHERE after the WITH | HAVING | | orderBy, limit, skip | ORDER BY, LIMIT, SKIP | ORDER BY, OFFSET, FETCH FIRST |

The examples use a piece of feedback on a docs page:

@NodeFragment({ labels: ['Feedback'] })
export class Feedback {
    @NodeId() id: string;
    page: string;
    helpful: boolean;
    createdAt: Date;
}

Group and count. "For each page, how many?"

const votes: Map<string, number> = await gom.groups.countBy(Feedback, 'page');
// Map { '/agents' => 2, '/install' => 3 }

Several numbers, and the three filters.

const ratings = await gom.groups.load(Feedback, (q) => ({
    where: q.createdAt.gte(march),                                // 1. which nodes are read at all
    keys: { page: key(q.page) },
    aggregates: { votes: count(), likes: count(q.helpful.eq(true)) }, // 2. which nodes count toward one number
    having: [['votes', 'gte', 3]],                                // 3. which groups are returned
    orderBy: [['votes', 'desc']],
    limit: 10
}));
// [{ page: '/install', votes: 3, likes: 2 }, …]: each row is typed by its keys and aggregates

| Filter | Removes | | --- | --- | | where | nodes, before anything is grouped | | a filtered aggregate | nothing: it decides which nodes that one number counts | | having | whole groups, after they are counted |

The mistake to look for is helpful.eq(true) moved into where. It is valid, and then votes counts only the helpful ones, so likes equals votes on every row.

Group by a node, not a value. "For each user, how many messages did they write?" A path says how a message reaches its owner, and the key comes back as the node, mapped as a load of its fragment maps it:

const top = await gom.groups.load(Message, (q) => ({
    where: q.role.eq('USER'),
    keys: { user: pathKey(GuideUser, [hop('HAS_MESSAGE', Direction.INCOMING, 'ChatSession'), hop('OWNED_BY')]) },
    aggregates: { messages: count() },
    orderBy: [['messages', 'desc']],
    limit: 5
}));
top.map((it) => [it.user.username, it.messages]);

The aggregates are count(filter?), sum(property, filter?), avg, min and max. having compares an aggregate with a number by eq, neq, gt, gte, lt or lte, its entries joined by AND.

What every engine agrees on:

  • A group exists because a node is in it.
  • A node that reaches no key node is in no group. One that reaches several is in each, and once in each, however many
ways lead there.
  • sum, avg, min and max skip a node with no value. Over no values sum is 0 and the others are undefined.
  • An aggregate's filter never takes a node out of its group: a group none of whose nodes meet it still has 0.
  • Ties are broken by the keys, ascending, so a top 5 is the same five on every engine.
Where Oracle differs: min and max of a text skip a value stored as "", which Oracle reads as no value. A sum or an avg of decimals is decimal arithmetic, so 0.1 + 0.2 is 0.3 there.

Saving Data

A save writes what the object holds. It doesn't depend on whether the object was loaded, or on anything the manager remembers, because the manager remembers nothing.

await gom.save(view);                                            // every field that is set; relationships are added, never removed
await gom.save(view, { replace: ['assignedTo'] });               // this field's list is the whole list
await gom.save(view, { replace: 'all' });                        // every relationship field's list is the whole list
await gom.save(chunk, { except: ['embedding'] });                // write everything but these
await gom.save(person, { only: ['name'] });                      // write just these
await gom.update(Person, id, (it) => { it.name = 'Ada'; });      // load, change, save what differs
await gom.create(person);                                        // make it only if it is not there; undefined when it is
await gom.edges.unrelate(nodeRef(Issue, a), nodeRef(Person, b), 'ASSIGNED_TO');

A save is one statement: the root, the relationships it drops, each related node and the relationship to it. It is applied whole or not at all, with or without a transaction, on every engine.

only and except name fields of the object, and for a view fields of its root. A view's relationships are written whatever they name. Give one or the other, not both.

save returns the saved object. On a class with a @NodeStamp field, use the returned object from then on: it carries the stamps the save left, on the root and on each related node that has one.

Relationships

A save adds the relationships the object holds and removes none. To remove, name the field in replace: its list is then the whole list.

  • A field removes only what it loads: relationships of its type and direction, to nodes with its target's labels.
  • removedTargets: RemovedTargets.DELETE_UNREFERENCED also deletes a removed target that nothing else refers to the way
the field did. The root is never deleted, nor is a node the object still holds in another field.
  • replace: 'all' covers every relationship field of the view itself. It is refused for an object that carries no
stamp, because the lists of an object built from scratch are not what the store holds.
  • A list to replace that is null or undefined is refused. An empty list removes every relationship of the field, and a
single-valued field that is null or undefined removes its relationship.
  • replace is refused, on a root that carries a stamp, if any relationship of the root was added, removed or changed
since the object was loaded. A save that only adds is not: two writers who loaded the same view can each add to it.
  • An UNDIRECTED field is met by a relationship stored either way. One is made, from the root, only when there is none.
  • A read-only field is never written: a @ReadOnly() field, a @GraphPath, a @Count or @Aggregate, and a list of
fragments read over several hops.

Load, Change and Save (update)

update loads the object, applies your change, and writes only what the change altered: the fields that differ, a field set to null, and for a view the relationships it added or dropped. Your change may alter the object it is given, or return a copy.

const saved = await gom.update(Person, id, (person) => {
    person.name = 'Ada';
});
  • If the node changed between the load and the save, update loads it again and re-applies the change: three attempts
by default (the fourth argument), then StaleObjectException. That needs a @NodeStamp field; without one a change by another writer goes unnoticed. On Neo4j the save compares the root's fields as they were loaded as well as the stamp: see When a Save Is Refused.
  • It returns undefined when there is no such node, or the node is deleted in between. It never makes a node.
  • A change that gives the object another id or another class is refused.

Create If Absent (create)

create makes the node only if no node has its id, in one statement. If one does, it writes nothing and returns undefined. With update, that saves an aggregate without losing any writer's data:

const stored =
    (await gom.create(proposition)) ?? (await gom.update(Proposition, proposition.id, (it) => it.merge(proposition)));
  • An object that carries a stamp was loaded, and is refused.
  • Two writers at the same moment can both find the node absent. On Neo4j, Memgraph and Oracle only a uniqueness
constraint on the id keeps the second from making the node again: uniquenessConstraint('Proposition', 'id'). FalkorDB runs one write at a time, so there the second always finds the first's node.

Null-Write Policy (NullPolicy)

await gom.save(chunk);                                       // IGNORE (default): merge-patch
await gom.save(chunk, { nullPolicy: NullPolicy.CLEAR });     // a full overwrite: nulls clear
  • IGNORE writes only fields that are set; a null or undefined field leaves the stored value. A partly-loaded
object never destroys stored data, an embedding included.
  • CLEAR makes the object authoritative: a null field clears its property. In JavaScript only keys the object holds
can be seen, so a declared field that was never assigned is not cleared. Use it with a complete object.

In a view the policy governs the root. A related node is written as under IGNORE.

Batch Save (saveAll)

const saved = await gom.saveAll(views);
await gom.saveAll(views, { replace: ['mentions'] });
await gom.saveAll(chunks, { only: ['summary', 'summaryModel'] });
  • Fragments go in one statement for each 1,000, of any classes and labels. A view is a statement of its own. Outside a
transaction of yours the statements run in one transaction, so a failure leaves nothing written; inside one they join it. FalkorDB has no transactions: see FalkorDB.
  • The objects come back in the order given, carrying the stamps the batch left. The batch writes the fragments' classes
in the order of their names and the fragments of each in the order of their ids, so two batches over the same nodes take their locks in the same order.
  • A batch that only adds doesn't check a fragment's stamp: a stale one is written anyway, and keeps the stamp it had, so
a later save of it is refused.
  • replace applies to views; a batch holding a fragment is refused with it. A stale view throws
StaleObjectException, and the batch then writes nothing.
  • With only or except, a node deleted since its object was loaded is made again. To write only where the node still
exists, use patchAll.

Conditional Write by Id (patch)

Write named fields of one node, by its id, if the node meets a condition, in one statement. The node isn't loaded: the store evaluates the condition under the node's lock.

const result: PatchResult = await gom.patch(Session, sessionId, {
    set: { lastActivityAt: activityAt },
    where: (q) => anyOf(q.lastActivityAt.isNull(), q.lastActivityAt.lt(activityAt))
});
result.found;    // a node has the id
result.applied;  // the condition held, so the fields were written
result.changed;  // a value differed, and the node's stamp was renewed

| Outcome | found | applied | changed | | --- | --- | --- | --- | | No node has that id | false | false | false | | The condition didn't hold | true | false | false | | It held, and every value was already as set | true | true | false | | It held, and a value differed | true | true | true |

On a view the fields are the root's, and the condition may name a relationship:

const narrated = (await gom.patch(MessageInSession, messageId, {
    set: { narration },
    where: (q) => q.session.sessionId.eq(sessionId)
})).applied;   // false for a message that is there, but in another session
  • set names fields to write, and clear fields to remove.
  • It never makes a node, isn't checked against a stamp, and hands none back. A node it changed has a new stamp, so an
object loaded before is refused when saved: load it again.
  • The id, the stamp, a bag's key, a @NodeLabels field and a related node's field can't be written.

Batch Patch (patchAll)

Write named fields of many nodes, where each is there, making none:

const results = await gom.patchAll(embedded, { only: ['embedding', 'embeddingModel'] });
const rewritten = results.filter((it) => it.found).length;

There is one PatchResult for each object, in order. only or except is required: to write every field, save. Under NullPolicy.IGNORE a named field that is null is left as the store has it.

Saves and Transactions

  • A save joins the transaction it runs in, a @Transactional() method's or runInTransaction's. Outside one it is its
own.
  • StaleObjectException leaving a @Transactional() method rolls the transaction back. A refused save has written
nothing; what the transaction wrote before it is undone only if the transaction rolls back.
  • A statement the engine turns away because another writer was changing the same node is run again, after a pause,
outside a transaction. Inside one it fails, and the transaction's owner decides.
  • A statement that lost its connection is not run again, though the Neo4j driver calls that retriable: it may have been
applied before the connection went, and a second run would find its own stamp and report a save that was applied as refused.

@NodeStamp: refusing a save when the node changed

Strongly recommended on any class that is loaded, changed and saved. It plays the part of JPA's @Version: optimistic locking, with a random value in place of a counter.

@NodeFragment({ labels: ['Person'] })
export class Person {
    @NodeId() id: string;
    name: string;
    @NodeStamp() stamp?: string;
}

The field is an optional string on a @NodeFragment, one to a fragment. A stamp is a node's, so it is refused on a view and on a relationship fragment.

The Stamp

The stamp is two random tokens of 64 bits each, stored on the node under __drivine.stamp as 3fa9c1d27b40e8a6:91d0f4b2c7ee5a13, whatever the field is called. Loading fills the field. Treat it as opaque.

  • The data token, the first, speaks for the node's properties and labels.
  • The relationship token, the second, speaks for its relationships, of every type, in both directions.
  • To mark a node is to replace one of its tokens. A save marks what it changes, and so must Cypher of your own.
Every node a save writes is given a stamp if it has none, whether or not its class declares a @NodeStamp field.

When a Save Is Refused

A save is refused when what it would overwrite changed since the object was loaded. Every save of an object that carries a stamp compares the data token. A save with replace compares the relationship token too. A refused save writes nothing and throws StaleObjectException.

| Since the object was loaded, another writer… | save | save with replace | update | |---|---|---|---| | changed a property or a label of the root | refused | refused | loads again and re-applies the change | | added or removed a relationship of the root, or changed one's properties | applied | refused | applied | | changed a related node | applied: the related node is written over the change | the same | applied | | deleted the root | refused, deleted is true | refused, deleted is true | returns undefined | | changed anything with Cypher that doesn't mark it | not noticed | not noticed | not noticed |

An object whose stamp is undefined isn't checked: it makes the node or overwrites it. The save is one statement that takes the node's write lock before it compares, so of several writers holding the same stamp exactly one succeeds.

On Neo4j a load can be handed a stamp newer than its data. Neo4j reads under read committed, without a snapshot, so a load that runs while another writer commits can find that writer's stamp and the data from before its write. update therefore compares each field of the root as it loaded it too, under the node's lock, and loads again if one differs: a field's value (a @JsonPacked field's as the text the node held), a @PropertyBag's keys and their values, and the labels a @NodeLabels field reads. A field the object doesn't hold and its class doesn't initialize isn't compared. A save of an object you loaded yourself has only the stamp to go on, and in that moment it can write over the other writer's change: where every writer's change must be kept, change the node with update. Memgraph reads from a snapshot, FalkorDB runs no read beside a write, and Oracle and Postgres read a statement whole, so on them the stamp is what was read.

try {
    await gom.save(edited);
} catch (e) {
    if (e instanceof StaleObjectException && !e.deleted) {
        // another writer got there first: e.foundStamp is what the node carries now
        await gom.update(Person, edited.id, (it) => { it.name = edited.name; });
    } else {
        throw e;
    }
}

Existing Data

A node written before stamps has none, and loads with the field undefined. The first save of it isn't checked and gives it a stamp. To stamp every node of a label at once:

await pm.execute(
    new QuerySpecification(MATCH (n:Person) WHERE n.\__drivine.stamp\ IS NULL SET ${Stamps.setClause('n')})
);

Cypher You Write Yourself

Cypher should mark what it changes: a write that doesn't is not noticed, and a checked save of an object loaded before it is applied over it. Stamps.setClause marks a node whose properties or labels you change, and Stamps.linksClause each end of a relationship you add, remove or change:

MATCH (p:Person {id: $id}) SET p.name = $name, ${Stamps.setClause('p')};

MATCH (a:Person {id: $a}), (b:Person {id: $b}) CREATE (a)-[:KNOWS]->(b) SET ${Stamps.linksClause('a')}, ${Stamps.linksClause('b')};

The property's name holds a dot, so Cypher that names it quotes it: ` n.__drivine.stamp . Cypher that replaces every property (SET n = $props) removes the stamp, and the next checked save of an object loaded before is refused.

What a Stamp Doesn't Cover

  • A related node isn't checked: save writes it whole. update writes only what the change altered.
  • An object with no stamp isn't checked, nor is a fragment in a saveAll.
  • One relationship token covers every relationship: a replace of one list is refused when a relationship of another
type was added.
  • Deleting a node removes its relationships without marking the nodes at their other ends.

@ReadOnly: Loaded and Never Written

``typescript @GraphView() export class IssueOverview { @Root() issue: Issue; @GraphRelationship({ type: 'ASSIGNED_TO', target: () => Person }) assignedTo: Person[]; // written @ReadOnly() @GraphRelationship({

... (README truncated for length)

NeshDevTech

NeshDevTech is a professional technology firm passionate about creating innovative, high-performance web and mobile solutions.

Core Services
  • Custom Laravel Apps
  • SPA (React & Vue.js)
  • Flutter Mobile Apps
  • SEO & Performance Opt
Get in Touch
Available for Projects

Need a high-performing system? Let's discuss your project.


© 2026 NeshDevTech. All rights reserved.

Built with

Chat with me