"Endless" suggestion list

[kelson42]

Current suggestion list can only deliver 200 suggestions maximum, even if we have 10.000. We could do better.

We should have an "endless" suggestion list which can be growing as long as the libzim/libkiwix delivers results. Basically the suggestion list could load blocks of (for example) 100 suggestions, and when the users attempt to reach the end of the list, then Kiwix would load dynamically the next 100 batch of suggestions. This until libzim/libkwiix can not deliver any result anymore.

[Chandra-Sekhar-Bala]

So, we have to use pagination in this right @kelson42 ?

[kelson42]

@Chandra-Sekhar-Bala No, this is not what is described

[Chandra-Sekhar-Bala]

So, at first we'll load 100 (block size) of data. And based upon the scrolling behavior, we'll load more blocks. This happens until data source exhausted - I guess it is right.
Also, I'm curious to know, how does this differentiate from paging?

[kelson42]

@Chandra-Sekhar-Bala Yes. UX is fundamentaly different, but not in principle.

[kelson42]

Possible in Qt looks like with something like https://doc.qt.io/qt-5/qtwidgets-itemviews-fetchmore-example.html, here the corresponding feature request for Kiwix Desktop kiwix/kiwix-desktop#413

[soCallmeAdityaKumar]

Anyone working on it ?Can I work on it?

[kelson42]

@soCallmeAdityaKumar No bu this might be difficult to achieve this without libkiwix12 introduction. Please wait a bit.

[kelson42]

Seems similar to #440, wonder if we should not close one of these as duplicate

[javaman97]

@kelson42 I hope we would have to use the "RecyclerView" to fix this issue. As soon as the user scrolls, the new list got recycles and updated as the user scrolls the screen until libzim/libkwiix can't deliver any more results.I hope I would definitely fix this issue.

[kelson42]

@javaman97 Would be great if you could implement it. That said I believe you might have to wait introduction of libkiwix12 to be able to request arbitrary suggestions range. Lets wait a few weeks.

[javaman97]

Thanks for having faith on me.I would be happy to help you.

[arexnnnn]

is this issue still open? i want to contribute.

[kelson42]

@arexnnnn The ticket is obviously open like you can see by yourself. What is your proposal to solve it?

[tanvijain13]

Hello @kelson42 I would suggest going with lazy loading for this issue, as we want to load when the scroller reaches to the end of let's say 100 suggestions, and then can continue loading more as we go until the library is out of suggestions. Can you confirm if this would be optimal, and I can start contributing to this if it is not taken yet!

[kelson42]

@tanvijain13 LGTM, go for it.

[MohitMaliFtechiz]

@kelson42, in PR #3216, we have incorporated the use of libkiwix version 12 binding. Additionally, the search functionality has been refactored. The libkiwix now returns a SearchIterator containing all possible results for a given searchTerm. For instance, if a user searches for 'Alpine', libkiwix may find 10000 results for this term, which are stored within the SearchIterator. However, directly loading this large dataset into the recyclerView at once is not efficient.

Currently, we are extracting all results from the SearchIterator and populating our recyclerView. To improve efficiency and ensure a smoother user experience, pagination is suggested as an ideal approach, as proposed by @tanvijain13. However, using the SearchIterator for pagination is not good IMO, since it would require keeping the entire object in memory throughout the pagination process.

To achieve efficiency and a smooth user experience. We can retrieve all the results from the SearchIterator and save them to a list, as we were previously doing. Subsequently, we load some initial data (e.g., 200 suggestions) from the list into the recyclerView. As the user scrolls to the bottom of the list, we fetch the next 200 items from the list and display them to the user, continuing this process as needed.

By adopting this strategy, we can balance efficiency and user experience while providing a seamless scrolling experience for users interacting with the recyclerView.

[kelson42]

@MohitMaliFtechiz The libkiwix searchiterator does not have all the results in memory. This is anyway not your problem as it happens in the libkiwix.

That said, this is indeed a good time to tackle this ticket. Please implement it like requested here: load only what is needed for a proper display and if user scrolls down the list, only then request more. Keep track in kiwix-android only about what has been displayed. Never request the whole resukt set.

[MohitMaliFtechiz]

This feature is implemented in #3217. Once we merged #3217 we can close this ticket.

[kelson42]

If I look to the PR it still says that it fixes only one ticket. Please use properly "Fixes #..." for each ticket it fixes.