Search issues because of outdated django-haystack&Whoosh and faulty order_by('id_order')

[ageron]

I have had many issues with the searches (the non-configured, out-of-the-box search functionality): sometimes the "total" would be >0, but the "rows" would be empty ([]), even though start=0 and rows=1000. Sometimes the result would be empty even though there should have been a result (I double-triple-quadruple-checked).

I upgraded django-haystack to 1.2.7 and Whoosh to 1.8.4 (I updated requirements.txt accordingly). This fixed most issues, but I still had the first issue (total>0 but rows==[]).

I ended up patching the execute_query function in commis/search/query_transformer.py to remove the order_by('id_order'). This fixes the above issue. I have not looked in details, but I wonder if the id_order field actually exists in all cases: this would explain why count() returns a value >0 (because it probably ignores the order_by clause, but no rows are returned.

Now I still have one remaining issue: searching for "recipes:mongodb::shard" returns an empty result, but searching for "recipes:*shard" works fine. This means that I cannot use many community cookbooks out of the box, including mongodb: I had to patch it to never use searches with : in them.

Hopefully, the above should provide work-arounds for most cases, except the : bug that I have not figured out.. yet?

[coderanger]

The ID order is in there because the results do need to be stable over time or you end up with problems. The Ruby and Erlang servers both use an opaque Solr ID friend for this. The exactly order is mostly irrelevant as long as it won't change over time.

[ageron]

Yes, by removing the order_by, I'm probably just trading a present bug for a future bug. I'm just identifying the source of the bug, but I have not fixed it.