Changes to search

[PeterJ]

Following some potential issues with search identified by [mention]StooB[/mention], we are making some small changes.

Previously you had to enter at least 3 characters in a search box (any characters including special characters). This has now changed to at least three alphanumeric characters. So from now on:

*** - Not accepted
D?? - Not accepted
Dizzy* - OK

At the moment this is only implemented in the main multi search entry box on the top right of the home page. I will implement it in all the advanced search fields in the next few days.

Please test and feedback here.

[R-Tape]

Ta!

Is there any way that e.g. "3D" could be a viable search?

[djnzx48]

Doesn't this prevent non-English language searches from working? For example, try searching for "ńńńńń" and it says you need "at least 3 alphanumeric characters".

[PeterJ]

Search is one of the most challenging areas especially when you are looking at non UK character sets as well. I'm very happy to share our code with any developers out there.

It may be possible to allow a small number of 2 character exceptions to the rule such as '3D'

This is what we are using, and we are trying to avoid allowing search returning huge numbers of rows:

https://www.php.net/manual/en/book.ctype.php

Peter

[djnzx48]

I'm not 100% sure this is correct, but would the following regex work?

^(?:.*[\p{L}\p{N}]){3}.*$

It looks for at least three Unicode letters or numbers in the string, and should work for other character sets as well.

[PeterJ]

I'm not 100% sure this is correct, but would the following regex work?

^(?:.*[\p{L}\p{N}]){3}.*$

It looks for at least three Unicode letters or numbers in the string, and should work for other character sets as well.

Thanks. We had looked at regex, but had gone for the PHP count commands. I will PM you our current PHP function and we can discuss.

[PeterJ]

Doesn't this prevent non-English language searches from working? For example, try searching for "ńńńńń" and it says you need "at least 3 alphanumeric characters".

Do you happen to know if those characters are
UTF-8?

EDIT:

Is this it?

https://www.fileformat.info/info/unicod ... /index.htm

[djnzx48]

Uh, I think so? I just searched and found it here: https://codepoints.net/U+0144. It was an example that I got from looking in this thread.

I don't actually have any PHP knowledge, but if I can be of any help then I'd be willing to try.

[PeterJ]

Thanks for the link.

Things like Extraña do actually work fine due to the work [mention]Einar Saukas[/mention] did.

What titles are you thinking about where the search would be problematic?

[djnzx48]

Well, I haven't found any proper examples yet since the majority of entry titles seem to be converted to the standard Latin character set. But if you just wanted to search for e.g. "aña", then you won't get any results. (And searching for it in the advanced search gives results for "ana" as well.)

[PeterJ]

Thanks for the example [mention]djnzx48[/mention].

Let me review, over the next few days. I won't implement on the advanced search until we have a resolution. I appreciate the feedback.

There are hundreds of posts on Stack Exchange about this sort of thing.

[PeterJ]

[mention]djnzx48[/mention]

Can you try it now. I have implemented the following:

https://github.com/infralabs/DiacriticsRemovePHP

Peter

[PeterJ]

Ta!

Is there any way that e.g. "3D" could be a viable search?

[mention]R-Tape[/mention]
Can you try it now with '3D ' (That is 3D or 3d followed by one space). Not perfect, but small steps...

[R-Tape]

@R-Tape
Can you try it now with '3D ' (That is 3D or 3d followed by one space). Not perfect, but small steps...

Excellent! Thanks.

[djnzx48]

It looks like it's gone back to the old behaviour now... as in input is unfiltered and wildcards can still be searched. That PHP script might work, but I suppose it would mean that non-Latin alphabets (such as Russian) could not be used? Not that there are currently any Russian titles in the database anyway.

[PeterJ]

Try again please.

[PeterJ]

[mention]djnzx48[/mention]

When you enter in a search now you will see three data fields above the search results:

Entry Text: aña
Processed Text: ana
Character Length: 3

These columns will be removed after the testing phase.

I'm not going to implement non-latin alphabets.

[djnzx48]

OK, yep. The Latin character replacement thing seems to be working. But it's still allowing wildcards for me.

[PeterJ]

OK, yep. The Latin character replacement thing seems to be working. But it's still allowing wildcards for me.

Try one more time for me please!

Entry Text: ***
Processed Text: ***
Character Length: 0

https://www.youtube.com/watch?v=FGBhQbmPwH8

[mention]R-Tape[/mention] & [mention]Joefish[/mention] - '3D' temporarily removed, will be back as soon as I know why it's breaking other things!

[hikoki]

Hi Peter. Could you add a Machine filter for All Classic Spectrum models? Thanks for the site.

[PeterJ]

Hi Peter. Could you add a Machine filter for All Classic Spectrum models? Thanks for the site.

Hi, yes. Good idea. Will add to the list.

[PeterJ]

[mention]R-Tape[/mention]

'3D ' now working again. Note - Uppercase D and space at the end. Apologies for the issues.

[Fahnn]

This probably isn't anything anything that anyone else wants (and if so, please just ignore me), but is it possible to search for games that have an entry in the "Comments" field? Some of the comments have some great information (e.g. Bandersnatch, Sqij!, etc.) and I'd love to be able to find more of that sort of thing.

[PeterJ]

This probably isn't anything anything that anyone else wants (and if so, please just ignore me), but is it possible to search for games that have an entry in the "Comments" field? Some of the comments have some great information (e.g. Bandersnatch, Sqij!, etc.) and I'd love to be able to find more of that sort of thing.

Thanks for the suggestion. I could do you a CSV or Excel export of all the games with comments, and include the comments. Would that help?

[hitm4n]

This is what we are using, and we are trying to avoid allowing search returning huge numbers of rows

Just throwing this thought in the ring...

If limiting number of results why not allow any old stupid search criteria but then limit what is returned to screen and post a message saying...

"This search generated more than 500 results, please refine your search criteria"

Or whatever number or message you see fit. Then all this work isn't needed?
Just an idea.