Where did nine records go?

This bug didn’t break a single button. The catalogue worked, search found things, filters counted. It was given away by one line in the agent’s report: “6,144 records remain in the development database”.

Where the difference came from

6,153 minus 6,144 is nine. Our export contains exactly nine records without a main title. The import met such a record, decided it was incomplete and silently skipped it. In the code it looked harmless: “Record without title skipped”.

Why that is badA record without a title is still a record. It may have authors, a call number, copies on the shelf. By throwing it away, the system pretended the book didn’t exist, while it was standing in the room.

How it should work

We already had the right tool for such cases: the “Data quality” section. It doesn’t hide problems; it turns them into a to-do list. A record without a title should be imported, shown as “[untitled]” and placed in the “No title” group, where the librarian can fix it.

We phrased the fix like this:

Import everything that contains any data at allOnly completely empty records may be skipped — and their number must be stated in the report.
Test on the original fileOur export must result in exactly 6,153 records.
Load the missing onesA repeat import of the same file adds the nine records to the “New records” group.

After the fix, the agent found all nine records and even pointed out the lines in the file where they were.

Katlog: the file dry run — 6,153 records recognised, nine notes about records without a title — The dry run of the original file finds all 6,153 records and lists the nine records without a title separately. The screenshot shows the Russian interface.

The lesson

After any import, compare the number of records in the file with the number in the system. It is the cheapest check there is, and it catches errors you can’t see in any window. That is exactly how the project began: the Aleph export first showed 5,858 records instead of 6,153, too.