Repository navigation
Fix unknown acquisition link type - #1385
Conversation
a68f199 to
5020f54
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1385 +/- ##
==========================================
+ Coverage 45.48% 45.55% +0.07%
==========================================
Files 58 58
Lines 5189 5196 +7
Branches 2680 2685 +5
==========================================
+ Hits 2360 2367 +7
+ Misses 1027 1026 -1
- Partials 1802 1803 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
veloman-yunkan
left a comment
There was a problem hiding this comment.
LGTM, save for one issue in the new test.
Book::updateFromOpds() threw for an acquisition link whose type is not one of application/x-zim, application/metalink4+xml, application/x-bittorrent and application/x-magnet. A single such link thus made the whole OPDS feed unreadable: kiwix-manage aborted on it, kiwix-serve refused to load the library, and so would any client reading a remote catalog that starts using a new link type. Such a link is now skipped, along with its length, with a warning on stderr; the book and its other links are kept.
5020f54 to
3a079c9
Compare
|
@hamazaspavetisyan I will merge this PR, but I'm really not convinced this is better with the PR than without. Please open an issue systematically before coding a PR, then we decide first if this is an issue. XML is machine readable and should be written by machine. The scenario you describe is not a scenario we have at Kiwix, therefore I don't understand why we spend ressources on it now. The fact that kiwix/kiwix-tools#844 has not moved for a month and is not ready is the problem! |
Given we've just introduce several new type for links, this PR can be considered allowing future flexibility without crashing. Also, given we use OPDS because it's standard, yet extend it with our own stuff, it's only legitimate to assume another OPDS feed could include our links in order to pass its feed to kiwix-serve without having to remove its custom links because kiwix-serve would crash. |
Problem
Since 069ef6c ("Parse acquisition-link type in updateFromOpds()"),
Book::updateFromOpds()reads thetypeof each acquisition link. It throwsstd::runtime_error("Unknown acquisition link type: …")for any type other than:application/x-zimapplication/metalink4+xmlapplication/x-bittorrentapplication/x-magnetBefore that commit, the type was ignored. Nothing in libkiwix catches the exception, so a single link with an unknown type makes the whole OPDS feed unreadable:
kiwix-manage library.opds showkiwix-serve --library library.opds--monitorLibraryreloadreadOpds()insideQtConcurrent::run)QFuture: the remote library is left with only the books before the bad entry, and the catalog never finishes loadingThis affects local library files, especially once OPDS is the only library format (#1377) and users write or convert such files by hand. It also affects remote catalogs: a catalog that starts using a new link type would break every client built with this code.
069ef6c is not in any release yet, so no released version is affected.
How to reproduce
Fix
In
updateFromOpds(), the call tofromMimeTypeToLinkKind()for a non-empty type is wrapped in atry/catch. A link of an unknown type is now skipped, together with itslength, and a warning is printed on stderr:The book and its other links are kept.
fromMimeTypeToLinkKind()itself is unchanged.With the reproduction file above:
kiwix-manage … showlists the book (exit 0);kiwix-serveloads the library.Tests
BookTest.updateFromOPDSSkipsAcquisitionLinkOfUnknownTypeTest: an entry with a link of unknown type (and its ownlength), followed by a ZIM link and a local path. It checks:ManagerTest.readOpdsKeepsBookWithAcquisitionLinkOfUnknownType: a two-book feed where one book has a link of unknown type.readOpds()succeeds and both books are loaded, with the affected book keeping its ZIM URL.