Improved Search: Search for short, long, common words

[macdjord]

Frequently when searching I get the message "The following words were not included in your search because they are too short, too long, or too common:". Now, I understand why this message appears: the search system maintains indexes of which posts certain words appear in; if a word is not indexed, it is discarded as a search key rather than run a linear search over the entire forum.

However, there is a better way. What the search system ought to do is first apply those search criteria which are indexed: other words, poster, date range, forum or thread restrictions. Then, as long as that narrows things down enough to be tractable (say, a few thousand results), apply the non-indexed search terms by linear text search.
The search system already does something like this when you search for a quoted multi-word string: since the indexes only ever cover single words, it has to be performing a text search after filtering by the indexed words. This functionality simply needs to be extended to non-indexed single words.

 

[alethiophile]

I agree with you that this is what the search system should be doing. However, to make it actually do that would presumably require source hacking, the which I am not in any real position to carry out at the moment.

I've seen mention of an improved-search function tied to a separate search engine. May look into this at some point in the future. However, without specific "install this extension" level pointers, I probably am not able to fulfill requests regarding search engine internals.

 

[macdjord]

It's possible that the current engine already supports this - as I mentioned, it's already doing this for multi-word searches - but it's simply not configured to use that functionality.