Every inherited Meteor app we open has a collection with three generations of documents in it. Some records have address as a string because that is how it was written in 2016. Some have address as an object because somebody improved it in 2019. Some have both, because the improvement was half-finished. The application code handles all three with a chain of if statements that nobody dares delete.
Schema definitions — SimpleSchema, Collection2, Zod, whatever you landed on — describe what a document should look like. They do not change the documents that are already there. The work that closes that gap is a data migration, and on a live Meteor app it is one of the few remaining places where people still write a script, run it by hand against production, and hope.
It is worth doing better than that, because the same machinery you build for mundane backfills is exactly what you need during a Meteor 3 upgrade and during a migration off Meteor.
The package situation, stated plainly
Most older Meteor apps use percolate:migrations — a small, well-designed package that stores a version number in a migrations collection and runs numbered up functions on startup until the database is current. It did its job for a decade.
It is also a Fibers-era package. Its migration functions were written to call Collection.update synchronously, and the package itself calls into the sync collection API. On Meteor 3 that does not work unchanged. The community has produced forks and successors (the quave fork is the one we see most often), and they are reasonable choices. Before you adopt one, check three things in the repository: a commit inside the last year, an explicit statement of Meteor 3 support, and whether the up function is awaited. That last one matters more than it sounds. If the runner does not await your async migration, it will mark the version complete the instant your function returns its unresolved promise, and your migration will be recorded as done while it is still halfway through.
The alternative is to own it yourself. A migration runner is roughly eighty lines: a collection of records keyed by name, a lock so two app servers do not run the same migration twice, and a loop that awaits each pending migration in order. For a single app with a known set of engineers, writing those eighty lines is often cheaper than tracking a dependency through the next upgrade. We have done both and do not have a strong religion about it. We do have a strong opinion about the properties the migrations themselves need.
Four properties that make a migration safe
Idempotent. Running it twice must be harmless. This is not a theoretical concern: migrations get interrupted by deploys, by a container restart, by someone hitting Ctrl-C when the output looked wrong. Write the query so that already-migrated documents are not selected — { address: { $type: 'string' } } rather than {} — and the second run becomes a no-op by construction rather than by luck.
Resumable. A backfill over four million documents will be interrupted. Iterate by a stable sort key and record where you got to, so a restart continues rather than starting over. The usual shape:
async function migrateBatch(afterId) {
const docs = await Customers.find(
{ address: { $type: 'string' }, ...(afterId ? { _id: { $gt: afterId } } : {}) },
{ sort: { _id: 1 }, limit: 500 }
).fetchAsync();
for (const doc of docs) {
await Customers.updateAsync(doc._id, {
$set: { address: parseAddress(doc.address) },
});
}
return docs.length ? docs[docs.length - 1]._id : null;
}
Nothing clever there, and that is the point. Note that it is updateAsync with an await in front of it. A missing await inside a migration loop is the most expensive version of that bug, because it turns a controlled batch of 500 writes into four million unbounded concurrent ones.
Reversible, or honestly labelled. Some migrations can be undone; write the down function for those. Many cannot — once you have parsed a string into fields and dropped the original, the original is gone. For those, say so in a comment, and make the forward migration write the new field without removing the old one. Deleting the old field is a second migration, shipped weeks later, after the code that reads it is gone. Two small migrations with a gap between them is the cheapest rollback story available.
Observable. Log the batch count, the elapsed time, and the remaining count on every pass. A migration you cannot watch is a migration you cannot decide to stop.
Batching is a pub/sub problem, not just a database problem
This is the Meteor-specific part, and it is the one that catches teams who have run backfills on other stacks.
In a conventional application, an unthrottled backfill costs you database CPU and some write contention. In Meteor, every one of those writes also flows through the reactivity system. Under Meteor 2, the oplog tailer reads each change and tests it against every live observer. Under Meteor 3, change streams do comparable work. If you rewrite a million documents in a collection that is actively published to connected clients, you are not just loading MongoDB — you are generating a million invalidations, re-running the matching publications, and pushing DDP messages to every browser with that collection on screen. We have watched a "harmless" overnight backfill take an app down at 2am because the batch size was "all of them" and four hundred open tabs were subscribed.
So: cap the batch size, sleep between batches, and run the backfill during a period you have measured rather than a period you assume is quiet. Watch observer counts and DDP message volume while it runs, not just database CPU. If the collection is heavily published and the backfill is large, consider doing the work against the database directly, outside the app process, where no observers are attached — accepting that clients will then see stale documents until their next subscription change, which is usually fine and occasionally is not. That trade-off is worth five minutes of thought before, rather than an incident review after.
bulkWrite through the raw collection is tempting for throughput, and it is the right tool for very large jobs. It also bypasses Collection2 validation and any before.update hooks your app relies on. Use it deliberately, with the knowledge that your schema layer is not watching.
Migrations during the two big transitions
During a Meteor 3 upgrade, your existing migration history is a liability you can mostly retire. Migrations that already ran on production will never run again. Rather than converting a hundred historical up functions to async, we freeze the old runner's history, record the current version as the new baseline, and convert only what is unrun. New migrations are written async from the start. The old files stay in the repository as documentation of how the data got here, clearly marked as not runnable.
During a migration off Meteor, this machinery becomes load-bearing rather than incidental. A strangler migration keeps the MongoDB data layer shared while the new application takes over routes one at a time, which means both systems read the same documents during the overlap. Shape changes in that window have to be additive and dual-read: write the new field, read either, remove the old field only after both applications stop referencing it. The backfill itself runs exactly as described above — the only difference is that two codebases, not one, have to tolerate the intermediate state. Teams that already write idempotent, batched, observable migrations find that this costs them almost nothing. Teams that have been hand-running scripts in a production shell discover the practice and the migration at the same time, which is a bad week.
The modest version of all this
You do not need a platform. You need a folder of migration files with names and dates, a runner that awaits them, a lock so two containers do not race, and the discipline to run every one of them against a restored production snapshot before it touches production. The cost is maybe a day of setup. The thing it buys is the ability to change your data model on purpose — which, on an application that has been in production for ten years and intends to be there for five more, is most of what maintenance actually is.
If you are looking at a collection with three generations of documents in it and an if chain nobody wants to touch, that is not technical debt in the abstract. It is a backlog item with a known shape.