Why does Google return hits based on the list of “Related” question titles, but not on the actual question content?

[Breck_Carter]

This is a question about how the SQLA website works, not about SQL Anywhere.

You can read the actual question over on the StackExchange support website here: http://meta.stackexchange.com/questions/5039/why-does-google-return-hits-based-on-the-list-of-relate...

I'm posting the link here in case someone in the SQLA audience has a clue about what's going wrong with Google searches...

...there are a lot of very smart people reading SQLA, and I need your help on this.

Is it Google's fault? ( fat chance on that 🙂

Is it StackExchange's fault? Have they messed up the sitemap?

Is it my fault, as SQLA administrator, for not doing something I should be doing? I have been told that it is not necessary for me to create a sitemap for Google, but I'm not so sure.

The bottom line is this: Simple searches using google.com don't work, neither do searches using the "custom search engine" (see "Search SQLA" at the top right).

And in case you haven't noticed, the official StackExchange search (in the grey link bar above) is really, truly, profoundly awful. Not sure, but it might be doing a SQL LIKE, it's that bad.

Anyway, if Google search can't be made to work, SQLA is doomed.

---

More discussion: I'm guessing here, but it's my impression there are two different algorithms in use by StackExchange: One algorithm matches entire questions and answers to come up with a "Related" list. That algorithm isn't very good. The list will not be missed IMO.

The other algorithm is used when you start a new Question and it matches your proposed Title with other titles to come up with the "Related Questions" list.

I think the second algorithm is wonderful... I often type in a title just to do a search, when I have no intention of actually asking a question.

To answer Volker's question in the comments, yes, I have just gone ahead with the display:none trick. It will take a very long time ( days, even 🙂 to determine if it improves (or even affects) the Google search.

One thing I am concerned about, however: Will Google interpret the mere presence of display:none as evidence of evil-doing on my part, and stop indexing SQLA altogether. Don't laugh! It is a huge topic of debate out there in SEO land.

The good news is, display:none is the technique suggested by a high-rep person on the StackExchange meta site. Just to be sure, I have asked the question here: http://meta.stackexchange.com/questions/5077/will-google-interpret-displaynone-as-evidence-of-evil-d...

[VolkerBarth]

Awful thing - I just tried for myself. And the silly StackExchange search for SQL Remote (without quotes) returns no hints, whereas for "SQL Remote" seems to return ALL questions. Quite useless:(

[VolkerBarth]

Don't know much about search engines, but AFAIK, this page's title is set correct to <title>Why does Google return hits based on the list of “Related” question titles, but not on the actual question content? - SQLA</title>, and there don't seem to be exclusion commands for search robots and the like.

[reimer_pods]

I found out, that the stackexchange search ignores search keywords at will: a search for "node address" listed all entries with address, while there's only one containing "node address". Seemingly it's an OR search, even if the search string is put in double quotes (no option 'search for all words'). But the Google Custom Search (Search SQLA) did find the matching entry. Strange thingy!

[Breck_Carter]

@Reimer: And now you know why I added the Google CSE thing, the builtin StackExchange (and StackOverflow) searches are just wastes of screen real-estate. I just wish Google searches actually worked properly... I tagged this question with careful deliberation.

[VolkerBarth]

@Breck: Have you checked whether this site gets according Google crawl attention (and if the robots.txt is fitting)? And is there a sitemap defined? (Don't get me wrong, I have nothing to do with such stuff so far, all my minimal "wisdom" comes from reading over the SX meta discussions - so I do not know how to check these things...).

[Breck_Carter]

@Volker: Since I don't own the site itself, I'm reluctant to try any form of "investigation"... there seems to have been many discusions of sitemaps and the like in the SX meta discussions, but it is interesting that my straightforward end-user-style question has had no answers yet, not even a comment.

[Breck_Carter]

@Everyone: One thought did occur to me... perhaps I could strip the "Related Questions" sidebar out since IMO it is pretty useless (too verbose, not targeted, similar in uselessness to those "tag clouds" that everyone thinks are so cool). BUT... that would only affect "wrong question being returned", not the problem of "right question not being returned".

[VolkerBarth]

@Breck: Are you going to test your own possible solution (i.e. "#related-questions { display: none; }")? (That may be the best attempt to find out whether this work for google's crawler or not...)

[Breck_Carter]

@Volker: see "More discussion" in the question above

[Breck_Carter · Accepted solution]

[placeholder answer]