At Linden Lab, we recognize that SL Search is an aspect of the Second Life experience that is a top “back to basics,” priority to improve. So, over the last few months, the SL Search team has been working hard to make Search results more accurate and prepare for the Teen Second Life migration to the Main Grid. Here’s a brief overview:

New Filters for Search with General or Moderate Ratings

Until now, we’ve had both land and language filtering in search results. However, there was no real difference between General and Moderate content filters. We’re tweaking the content filters, both by defining a “Moderate” set of keywords and by altering some of the teen-related language in the adult filters. If your parcel or profile does not appear at the maturity level you expect it to, then edit your text. You will usually see your edits reflected in Search within six hours.

Classified Ads Now Get Extra Level of Rating Validation

Classifieds have always allowed Residents listing a Classified to self-declare the maturity rating of their ad. With teens gaining access to the Main Grid, we’ve added content filtering to provide another level of rating validation. If you list an ad and don’t see the maturity level you’re optimizing for in search results, then just edit your text. You should see the edits reflected in Search in about an hour.

New Functionality Automatically Builds in Keyword Variations

We’ve also added new functionality (stemming and stop words, for all the industry professionals out there) into Classifieds, which means that advertisers don’t have to stuff their descriptions with every variation of a keyword. For example, a search for “Dance” will return descriptions with “dancing,” “dances,” and “danced.”

More Sophisticated Filters to Identify Spam and Optimize Search Results

We’ve moved from the more rigid “past this point and you get dinged” method of calculating boost to one that balances several features of your page. These aspects include how often words are repeated, the nature of the objects on your parcel, the size of the parcel, unique descriptors, and the use of symbols and ALL CAPS. It is important to note that the filter takes into consideration many things. Depending on the combination used, your score could be impacted negatively or not at all.

A note about the perceived “size boost”: We want to promote all relevant locations in Second Life — big or small. Larger stores, with bigger inventories, are more likely to be mistakenly downgraded in Search for repetition in their descriptions and objects. So, we give larger parcels a minor buffer to allow for that. We’ve also adjusted this buffer so that it should be less noticeable. In addition, we now apply a negative boost for parcels that are below the minimum size permitted in search.

All of these factors are now rolled into the one visible spam boost rating on each parcel. It’s worth checking it over the next two weeks as we roll out changes and make the appropriate updates to your information. Edits are typically reflected on the World page in a few minutes and up to six hours in Search.

Destination Guide

Finally, we’ve made some changes to the Destination Guide to reflect each listing’s parcel maturity. This will ensure that teens have access to all of the Destination Guide listings they see in the Find window and all Residents will now see Destination Guide entries filtered by maturity level.

Let Us Know What You Think

We hope that these changes benefit everyone in Second Life — inworld businesses and Residents alike. Keep an eye on Search release notes for ongoing updates. Try it out and let us know if you notice the improvements. We look forward to your feedback in comments below!

This morning, we had a small release that we believe will make a big impact on the quality and consistancy of Search. We have moved from a crawl system to a feed system. In our previous crawl mode, the GSAs handled discovery and indexing inworld data with its own schedule and rules. With the new feed system, we generate and push content to the GSAs for indexing and have more control over when it is updated.

Please see our release notes!

Expect that ranking will be changing, and will settle by end of day. We are also publishing an update to the Search FAQ shortly and I will update this post as soon as it is published!

I’m feeling a little A.A. Milne this morning, so forgive the title.

Early this week, the additional relevancy improvements associated with last week’s release finished rolling out. The index update took too long and we’ve made changing that a priority. It’s a huge frustration that we share with Residents.

There were a couple of high-profile fixes in the last update. First and foremost, we have resolved WEB-2378  (Web and Inworld Search results are being mixed). This one was affecting relevancy for a number of keywords and we’re glad to get the fix out! Next, we have, WEB-1819  (Find is resetting to start page). Having a way to easily get back to a previous Search is central to basic usability. The Search team came up with a solution that we think fits that need without interfering with the core search experience. We know some people would prefer a different solution, but we hope you’ll give this a chance while we iron out the bugs.

Unfortunately, we also found some bugs pretty quickly in the last release and we’ve scheduled another deployment next week to release the fixes for these. Namely, they are that exact phrase searching is no longer possible and WEB-2536, a problem in our spam filter changes. In addition to that spam issue, we also see some enterprising Residents trying to spam in some pretty creative ways to get around the filter and we’ll be looking at those, too. Expect lots of upcoming tweaks!

Residents have all taught me to be explicit about what else we’re working on, so forgive me if there is a little repetition here. First off, we’re replacing the maturity ratings system. This is a multi-phase project that we are testing at each step.

Let me segue into a side note about the current maturity system. Surprising as it may be, the current maturity system is working as intended. It is counter-intuitive, but it’s not broken. The GSA looks at each set of results it’s been given independently and it evaluates relevancy based on the set it has. Practically speaking, this results in some odd behaviour, but it also results in some expected behaviour.  So, I wanted to write a little bit about it.

First, the larger the data set, the less likely your parcel will be to show up highly ranked for your keywords. I know, this one makes sense. Adding more maturity levels generally means that your parcel is being evaluated against more parcels that might be similar. The GSA, just like Google, only returns the first thousand results. If you aren’t there, try refining your keywords. instead of optimizing for “beach” to be your main keyword try something more relevant to your business, for example, “beachfront rentals”. The more specific queries will naturally have fewer relevant results. I have been a Search professional for a decade and I have never seen such an enormously long tail of search queries as I have seen in Second Life. Residents are using extremely specific queries!

Second, we realize that the current system is too unpredictable and frustrating for residents. Our new system will eliminate the counter-intuitive behaviour as well as extending smarter maturity ratings to Classifieds, Events, and database sorting.

Other fixes included:

• Tweaks and fixes to the html pages in web Search
• Search team’s part of the fix for owner links breaking in parcel descriptions
• Lots of small localization fixes and updates
• Stemming refinements to increase relevancy
• Javascript warning enabled for residents who disable it.
• Query speed testing and optimization
• Lightened scroll bar in viewer Search
• Spam system update

Next up, in additon to the items I already mentioned, we have a number of improvements on deck. Currently, Classifieds ads only count a portion of their actual clicks and teleports, so, the fix for that is on its way along with a number of localization changes, and some work on the new events browse tab. In addition, we’re testing what we hope is a vastly improved way to roll out changes. This new deployment change won’t be live with our release next week, but we’re testing it and focused on getting it out as soon as possible.

Thanks again for your comments! Sadly, I can’t enable them this week, but I’ll ensure we have them up for the Search post next week!

This  morning, you should continue to see all the improvements associated with this week’s release rolling out. It’s not complete yet, but we’re monitoring it over the weekend to ensure it goes out as expected.

In addition, we’re exploring other options for rolling out changes. When we do a big update, there are 1.5 million documents to reindex and process, this doesn’t happen fast, and it doesn’t even happen overnight. The Search team is actively planning ways to reduce that time so, when things do change, they change quickly, without the odd, in-between state of the index. We plan to have this in place for our next release.

Thanks for your patience!

We  had a small release last week and have been fine-tuning performance in  preparation for our next release. Improving performance is a major  concern of ours and if you haven’t yet, please download the latest Viewer. Search performance depends, in part, on Viewer  performance and every Viewer release improves Search performance.

Just released!

You may have already noticed this additional option appear on your in-viewer Search last week. If not, check it out! You now have the option to browse events straight from the Find window in Viewer 2.

Some features of note:

1) Featured
The  editorial team at Linden has stumbled across a few events we wanted to  bring your attention to, so we’ve included one of their selections up  top. If you have an event worthy of notice, please send in your  suggestions!

2) Choose a date
By  default, you’ll see events listed starting with the top of the current  hour. If you’re planning ahead, we have a convenient drop-down calendar  for you to choose another date.

Browse by interest
3)  You also have the option to simply select an Event category. We’re looking to expand categories as well and we welcome your suggestions.

Additional fixes included:

• Performance tweaks
• Teleport not triggered for specific locations in viewer search
• Changes to filtered words (Which included this fix WEB-2023)
• Multiple repeat Destination guide entries on web search
• Profiles missing images

Release next week!

The  fix for WEB-2378, finally, is coming along with a number of tweaks to the world pages, like visible html tags, table misalignment, classifieds tracking, and broken group  links. Also coming next week, is an updated FAQ with additional information on how Search works, including suggestions for all of you who would like to appear higher in listings for your keywords.

We’re working hard to increase the frequency of releases and enable the Search team to deploy when needed in order to fix bugs in a more timely manner. Thanks for your ongoing patience and understanding.

On a related note, thank you to everyone who is talking about Search  publicly, in social media, comments, and in forums everywhere. It’s a tremendous resource and we value your input highly. Please keep it up!

We intended to have a Search release incorporating a number of improvements in June. We did have one, sort of. During the release, we became aware, both internally and through Jira, that we had another scenario to take into consideration. So, we made the decision to roll back the release. Since then, we have fixed the Jira issue, dealt with the performance implications, and made some high-value additions to the release.

Our main focus for this next release is getting rid of our legacy Search system, one that has been patched, updated, and added to for years.  We’re replacing it with a new, faster, more dependable way of generating the Search indices. In short, it will be significantly improved. In addition, we wanted to tackle the two main issues we highlighted in the last post. Namely, a significant bug that caused us to drop parcels and secondary information that was causing bad Search results. It’s this second point I’d like to expand on a bit here, before our next release is out.

The Association Problem
When I first joined the lab in December, I was made aware of a report from a resident land owner whose region was somehow associated with the word “Nazi” in Search. There were no WWII role-playing events and no German history groups associated with his land, so he was justifiably unhappy about this. After investigating thoroughly, we narrowed it down to a picks problem. Some Residents, a literary group inworld, who had chosen his place as a pick had phrases like “grammar nazi” in their profiles. We contacted the 2-3 Residents about the issue and asked them to change their descriptions or delete this particular pick. Unfortunately this didn’t solve the problem because the GSA had “learned” that his region had something to do with Nazis. It took our GSA6 upgrade and fresh Search indices for this problem to finally go away. This is one of many examples; we’ve heard from several Residents citing their own favourite examples of this issue.

More Picks Issues
Aside from these accidental association issues, we have three main other issues with the picks data in Search. First, some enterprising individuals have actually created businesses based on paying people to influence Search with picks.  Second, Residents have been using picks to highlight those they care about, their friends and family in Second Life, rather than just places.  I think it’s nice to call out your important relationships in a public way, it’s just not good data for Search. Third, places frequently come and go in Second Life and many picks are outdated. There are those who like to keep a pick of a former favourite spot as a memory, but often, people simply don’t realize that their pick isn’t accurate. In either case, it’s more bad data from the search perspective.

These cases add to the problem of using picks data in Search and chasing the consequences of these issues as they come up is a time-consuming pursuit that doesn’t actually address the core issue. Though we think picks can be valuable information on important inworld spaces, right now, picks are a problem in Search.

We’ll continue to test different and better ways of using picks information, but for now, starting with this upcoming release, picks are no longer included in Search relevancy. They will still be visible in resident profiles in the viewer, but they won’t affect Search until we find a better way to use that information and add value to Search. We do appreciate the value of picks, but when we do include them again we want to be sure it adds to the relevancy rather than making it worse.

I look forward to writing more this week about the rest of the improvements included in the upcoming release!

Following  on from the previous blog post discussing the work we’re doing to fix search – today we’re happy to announce the full release of our fixes and improvements.

Perhaps the most important part of today’s release is the fix for the bug that was causing some parcels  to be dropped from the search index The fix involved replacing legacy components of the Search system which has had the added benefit of  greatly improving the speed at which the Search index gets updated. In other words, changes you make will be reflected by Search much more rapidly! This new system also brings us the increased dependability and consistency we need for Search.

Index Generation

Until this week, updates could take 72-96 hours to fully propagate in Search.

This meant that new parcels and regions would take a while to  show up in Search and merchants editing their search listings were not  able to monitor their changes in a timely fashion. By comparison the new  system updates in under 12 hours! (Plus, we plan to cut that in half in the upcoming weeks!)

Spam

Also in  this release are improved methods to deal with Search spam, starting  with keyword stuffing (using large numbers of words, often the same or very similar and often not related to the location itself). You should  therefore notice that fewer of the top ranking results will look like  they were written solely to game the rankings. Increasingly, if you add  excessive numbers of keywords this may count against your ranking. Try  to use fewer and more accurate keywords, including key phrases, to  increase your relevance. If you would like to refresh your memory on  best practices, please read our guidelines.

Improvements and Fixes List

More improvements include:

    ◦    Improve search listing page generation

     ◦   Increase crawl update speed and reliability

    ◦    Remove picks data

    ◦    Increase search index size

    ◦    Adjust column headers for events

    ◦    Exclude parcels for  private sale in general search

    ◦    Activate arrow keys for  browsing Viewer results

Bug fixes include:

    ◦    Results  appearing with “None” text

    ◦    Eliminate misleading result  count estimates

    ◦    Re-index groups missing from Search

     ◦   Eliminate English spelling suggestions for non-English users.

     ◦   Encode special characters in V2 results

More Improvements ahead