Repeat imports without fear
The catalogue is already in the new system and the librarian has corrected hundreds of records. Now a fresh export with new acquisitions needs to be loaded. How to make sure not a single correction is lost.
The first import is simple: an empty database, load everything. A repeat import is one of the hardest problems in the whole system. The database already holds corrected records, deleted records, items on loan. And the new file is a mix of old and new.
The core principle
To make this work, the system keeps two versions of every record: the current one and the last one loaded from the source. On a new import it compares each field:
- unchanged everywhere — do nothing;
- changed only in the source — may be updated;
- corrected by the librarian — keep the librarian’s version;
- changed in both — show a conflict with both values side by side; the librarian decides.
Groups in the dry run
- New recordsNot in the catalogue. Default: add.
- Confident matchesThe record already exists. Default: update while keeping corrections, and add new items.
- Needs a decisionThe match is uncertain — manual choice only.
- DeletedRecords the librarian has deleted. Skip.
The import never touches items with requests or loans. And the whole last import can be undone within 24 hours.
How it was improved
At first, when the very same file was loaded again, the system put 1,275 records into “needs a decision”, although they should have matched confidently — and later the dry run even began to crash. After the rework, the same file gives a clear picture: 9 new records, 6,144 confident matches and not a single uncertain one. Loading it again creates no duplicates, and manual corrections are kept. How this was done is told in “22 fixes in three stages”.
And that is a lesson too: a cautious error is better than a bold one. An extra question annoys the librarian, but two different books silently merged into one would be far worse.