Introduction
__Tymly is the product of many inter-related Node.js packages. The source code for all these packages is maintained in separate GitHub repositories in the WMFS organization.__
However, for those wanting to develop the Tymly framework itself, it can be tricky to keep-track as new Tymly repos are added and existing Tymly repos evolve. It's also important to ensure all these repos are linked together locally for the best possible developer experience.
_And that's what this repo can help with!_
Here we have an ___empty___ pnpm workspace-powered monorepo.
By following the instructions below, it's possible to automatically fill the empty /packages, /plugins, /blueprints, /cardscript, /apps, /mods and /e2e directories with the freshest Tymly code from https://github.com/wmfs.
- __Subsequent synchronization attempts will update local repos as necessary, and clone anything new that's become available.__
- __Repos you're working on are safe:__ if you've checked out a branch, sync won't touch it. It just tells you what's new on
master.
Environment
__There are a couple of things you'll need installed for all this to work...__
Git
Git is a version-control system for tracking changes in computer files and coordinating work on those files among multiple people.
- https://git-scm.com/downloads
Node.js
Node.js is an open-source, cross-platform JavaScript run-time environment that executes JavaScript code outside of a browser.
- https://nodejs.org
- Node
v20or later is required.
pnpm
We use pnpm workspaces to link together all the various Tymly packages. Shared dependencies are stored once on disk and hard-linked into each package.
With Node installed, install pnpm globally:
`` bash
npm install -g pnpm
Then check it's working with pnpm --version, which should print 12 or higher. Lerna is no longer needed.
> If you have Corepack enabled, it may intercept the pnpm command and fail with a Cannot find module ... pnpm.cjs error. Run corepack disable pnpm and install pnpm with npm as above.
GitHub Access Token
> Instead of using your GitHub password, we use a Personal Github Access Token. > This approach provides a few advantages - in particular finer access control and more specific monitoring. __Be sure to keep the value of your Access Token value secret!__
__Assuming you've already signed-up with GitHub, you'll need to create a new Access Token for all your Tymly-related interactions.__
- Generating a new token value is easy enough, first go here:
- Then click the __"Generate new token"__ button.
- As for those __Scopes__, click this one:
- Then hit __"Generate token"__.
- Copy the token value: we'll be setting an environment variable to it later.
organization's repositories.
Git Credentials
__The token above is only used to ask GitHub which repos exist. Cloning and fetching use your own git credentials__, so git needs to be able to reach private
wmfs repos without asking for a password:
- On Windows and macOS, Git Credential Manager (included with Git for Windows) handles this. Signing in once is enough.
- Alternatively, if you use the GitHub CLI, run
gh auth setup-git.
Sync runs git without interactive prompts, so if your credentials are missing, the affected private repos are reported as errors rather than leaving sync waiting for a password.
npm Access Token
__Many
@wmfs packages are private on npm, so installing them needs an npm access token.__ You'll need an npm account that's a member of the wmfs organization. Ask a team member to add you if you're not.
> __Not part of WMFS?__ You won't be able to install private packages, so you don't need an npm token. Use
pnpm sync:public-only, which only syncs public packages.
- Sign in at https://www.npmjs.com, then open __Access Tokens__ from your profile menu.
- Click __"Generate New Token"__ and choose a __Granular Access Token__.
Give it a name like "Tymly Monorepo*", and set the expiry to the longest allowed (90 days).
- Give it __read-only__ access to packages, and no access to organizations. Installing doesn't need anything more, so don't use a token that can publish.
- Generate the token and copy its value.
Then add it to the .npmrc file in your home directory (create the file if it doesn't exist):
//registry.npmjs.org/:_authToken=npm_yourTokenHere
ERR_PNPM_FETCH_404 GET https://registry.npmjs.org/@wmfs%2Fsome-private-package: Not Found - 404Check it works withnpm whoami, which should print your npm username. __Keep the token secret__, and never commit it or put it in the repo's own.npmrc.When your npm token expires
npm tokens expire after 90 days. When that happens, installs fail with a misleading error, because npm answers __404 Not Found__ for private packages rather than "not authorised":
cd tymly pnpm install__If you see a 404 for an@wmfspackage, check your token before anything else.__ Create a new one as above, and replace the old line in~/.npmrc.pnpm syncandpnpm bootstrapcheck your npm token before doing anything, so an expired token stops them straight away with a clear message instead. You can run the same check yourself withpnpm check-npm.TYMLY_NPM_TOKEN_EXPIRESThey also warn you when your token expires within 14 days, so you can replace it in good time. The expiry date is read from npm. If that isn't possible, you can record it yourself by setting the
environment variable (see below). It's also worth setting a calendar reminder when you create a token.git clone https://github.com/wmfs/tymly.gitCloning
Next you'll need to clone this repo. From the __Git__ shell:
Then, from the command prompt, install all the __Node.js__ packages required to make this repo work:
Downloading repo information from https://github.com/wmfs Cloned plugins/tymly-rbac-plugin On branch plugins/tymly-pg-plugin (feature/connection-pool, master has 1 new commit) Updated plugins/tymly-core (pulled 1 commit)# Environment VariablesTYMLY_GITHUB_TOKEN__To integrate this repo with your GitHub account, an environment variable will need defining. A second one is optional...__
| Environment Variable | Notes | | -------------------- | --------------------------------------------------------------------- | |
| And the value of this environment variable should be set to the personal __Access Token__ you previously generated. | |TYMLY_NPM_TOKEN_EXPIRES| Optional. The date your npm token expires, for example2027-02-01. Only needed if sync can't read the expiry from npm; see When your npm token expires. |/tymlySynchronizing
Nearly there! :smiley:
To synchronize your empty Tymly monorepo, run this from within the
directory:pnpm syncTYMLY_GITHUB_TOKEN__Which will:__
- Connect to GitHub (using the token in
) and find every Tymly repo in the WMFS organization.masterClone any you don't have yet into the right directory. Update the ones you do have, where it's safe to: * __Onwith no local changes:__ fast-forwarded to the latest commit. * __On another branch:__ left exactly as it is. Sync fetches, then tells you how many new commits are onmasterso you can merge or rebase when you're ready. __Uncommitted changes, unpushed commits, or a merge or rebase in progress:__ left alone and listed under Needs your attention*.__...which should lead to output looking similar to:__
Cloned (1) plugins/tymly-rbac-plugin Plugin for Tymly
Updated (1) plugins/tymly-core 1 new commit ff3037f fix: handle missing state machine (Jane Smith, 2 hours ago)
Needs your attention (1) plugins/tymly-pg-plugin On feature/connection-pool: master has 1 commit(s) you don't 8ccdfe1 feat: support schema search paths (Jane Smith, 3 hours ago)
1 up to date, 1 cloned, 1 updated, 1 need attention, 0 errors
Dependencies may have changed. Reinstall/relink before running anything.
Done (14 seconds).
Running pnpm install...
json "config": { "tymly": { "type": "blueprint" } }__Justpnpm syncanytime you want to ensure your local Tymly repos reflect those on GitHub.__pnpm sync:dryOptions
| Command | What it does | | ------- | ------------ | |
| Shows what sync would do, without changing anything. | |pnpm sync:verbose| Also lists every repo that was skipped, and why. | |pnpm sync public-only| Only syncs public packages. | |pnpm sync --no-install| Syncs repos but skips thepnpm installat the end. | |pnpm sync --concurrency=4| Works on fewer repos at once (the default is 6). Useful on slow connections. |pnpm sync --dry-run public-onlyOptions can be combined, for example
.@wmfsIf anything fails, sync finishes the other repos, lists the failures under Errors, and exits with a non-zero code.
If the final install fails with a 404 for an
package, your npm token has probably expired. See When your npm token expires.package.jsonWhich repos are synced, and where
A repo is synced if its
hastymlyin itskeywords. The directory it goes into comes from its other keywords:e2e| Keyword | Directory | | ------- | --------- | |
|/e2e| |mod|/mods| |app|/apps| |cardscript|/cardscript| |blueprint|/blueprints| |plugin|/plugins| |package|/packages|package.jsonIf a repo has more than one of these keywords, the one highest in the table wins. To choose the directory explicitly instead, add it to the repo's
:
json "config": { "tymly": { "sync": false } } `To stop a repo from being synced at all, set"sync": falsein the same place:
Linking and installing
After syncing, lerna-sync may show this message:
Dependencies may have changed. Reinstall/relink before running anything.
You don't need to do anything about it:
pnpm sync finishes by running pnpm install, which links every synced package to its local siblings and installs everything else. It's incremental, so it's quick when little has changed. (It's skipped for --dry-run and --no-install.)
Local Tymly packages are always linked in preference to published versions, even when version ranges don't match. See
.pnpmfile.cjs for how.
If your install gets into a strange state, start from scratch with:
pnpm bootstrap
which checks your npm token, then deletes every
node_modules directory and reinstalls.
Other useful commands:
| Command | What it does |
| ------- | ------------ |
|
pnpm test | Runs test in every package that has one |
| pnpm --filter @wmfs/tymly test | Runs test in a single package |
| pnpm -r --workspace-concurrency=1 test | Runs tests one package at a time (useful if suites clash over a shared database) |
| pnpm clean | Deletes every node_modules directory |
| pnpm check-npm` | Checks your npm token works, and warns if it expires soon |
Next steps
__With your Tymly repos cloned and packages installed, what next?__
- Check out our Tymly docs site.
- Have a read of our contributor Code of Conduct.
- Also, please read our notes about contributing.