Saturday, December 27, 2008

Japanese WMC Blog launched

Konnichiwa! Hajimemashite! *Hello, Nice to meet you!

We just launched a new Webmaster Central Blog in Japanese. For those of you who feel more comfortable reading Japanese, and are interested in webmaster-related information from Google, and even learning about issues specific to our region and language, we hope you enjoy on our Japanese version of the Webmaster Central Blog :D

Thursday, December 25, 2008

Feliz Navidad from the Spanish Webmaster Central team!

About three and a half months ago we kicked off the Spanish Webmaster Central blog hoping to reach many webmasters. Given the time of the year, we would like to say a big ¡Muchas Gracias! to all our readers.

It's been both a pleasure and a great opportunity for us to share our knowledge and hear your feedback. A few of this year's highlights:

For the blog, we had:
As for conferences, we had the chance to talk to some of you in:
And last, but not least, the Spanish Help Group, with lots of interesting stories.

This is us, several members of the Spanish Webmaster Central team:

From left to right: Cristina, Alvar, Rebecca, and Esperanza in Google's Dublin office, with a holiday touch :)

Tuesday, December 23, 2008

Wishing you and your site a happy holiday!

Your presence is our favorite present -- thanks for joining us for another year of making your site, and therefore the web, a better place.

Every day we see new people commenting and joining the discussion. This holiday season we'll try to update our blog to accommodate your growing needs. Always feel free to let us know how we're doing (especially if we publish a typo! :), because first and foremost and everywhere in the middle, we're trying to improve for you.

Happy holidays from all of us at Webmaster Central.

Monday, December 22, 2008

Quick and easy tips for the holiday rush

Season's greetings, webmasters! We've compiled a list of quick and simple tips for websites preparing for the holiday rush. For online and offline retailers, we understand that your website is a big part of your business, especially this time of year. Whether it's to make the sale online or to increase foot traffic to your brick-and-mortar location, your web presence is a critical part of your business plan. The tips below are fast, free, and can make a big difference.

Verify that your site is indexed by Google (and is returned in search results)
Check your snippet content and page titles with the site: command [] -- do they look accurate and descriptive for users? Ideally, each title and snippet should be unique in order to reflect that each URL contains unique content. If anything is missing or you want more details, you can also use the Content Analysis tool in Webmaster Tools. There you can see which URLs on your site show duplicate titles or meta descriptions.

Label your images accurately
Don't miss out on potential customers! Because good 'alt' text and descriptive filenames help us better understand images, make sure you change non-descriptive file names [001.jpg] to something more accurate [NintendoWii.jpg]. Image Search is one of our largest search properties, so you should take advantage of it.

Know what Google knows (about your site)
Check for crawl errors and learn the top queries that bring traffic to your site through Webmaster Tools. See our diagnostics checklist.

Have a plan for expiring and temporary pages
Make sure to serve accurate HTTP status codes. If you no longer sell a product, serve a 404. If you have changed a product page to a new URL, serve a 301 to redirect the old page to the new one. Keeping your site up-to-date can help bring more targeted traffic your way.

Increase foot traffic too
If your website directs customers to a brick-and-mortar location, make sure you claim and double check your business listing in Google Local.

Usability 101
Test the usability of your checkout process with various browsers. Ask yourself if a user can get from product page to checkout without assistance. Is your checkout button easy to find?

Tell us where to find all of your web pages
If you upload new products faster than Google crawls your site, make sure to submit a Sitemap and include 'last modification' and change frequency' information. A Sitemap can point Googlebot to your new or hard-to-find content.

Manage your sitelinks
Your site may be triggering Sitelinks in the search results, so check the links and make sure the destination pages are fully functional. Remember: in Webmaster Tools you can remove any sitelinks that you don't think users will find useful.

Don't forget to check out these additional resources:

Thursday, December 18, 2008

Sitemap Submission Made Simple

Submitting a Sitemap to Google just became even easier. No longer do you have to specify the Sitemap file type—we'll determine the type of data you're submitting automatically. Let's take a quick look at the kinds of Sitemap files we support as well as the ways they can be submitted to us.

A sample Webmaster Tools account with Sitemaps

Sitemap file formats supported by Google

Part of what makes the web so interesting is that there are so many different kinds of content out there. Do you use videos on your website? If so, send us a Video Sitemap file so that we can send you visitors to those videos! Do you host source-code samples? Submit a Code Search Sitemap! Here are the various kinds of Sitemap files that Google supports at the moment:

If you have multiple Sitemap files that you wish to submit to Google, you can include up to 1,000 of these in an XML Sitemap Index file. If you have more than 1,000 Sitemap files, you can just submit multiple Sitemap Index files - we'd love to take them all!

Submitting your Sitemap files to Google

Once you have your Sitemap files ready and available on your server, all that's left is making sure that the search engines can find them. Google supports three simple ways to submit Sitemap files:
  • Using Google Webmaster Tools
    Submitting your Sitemap files through Google Webmaster Tools is the preferred way of letting us know about them. The main advantage of doing it this way is that you'll always have direct feedback about how your Sitemap files were downloaded (were we able to reach your server?), how they were recognized (were they in the right format?) and what happened to the web pages listed in them (how many were indexed?). To submit your Sitemap files, make sure that your website is verified in Webmaster Tools, then go to "Sitemaps" in Webmaster Tools and enter the file name of your Sitemap(s).

    Sometimes it makes sense to keep your Sitemap file on a different server / domain name. To submit Sitemap files like that, you must verify ownership of both sites in Webmaster Tools and submit the Sitemap on the appropriate site. For instance, if your Sitemap file for is kept on then you need to verify ownership of both sites and then submit the Sitemap file under (even though the URLs listed in it are for For more information, please see our Help Center topic on submitting Sitemap files for multiple sites.
  • Listing Sitemap files in the robots.txt file
    Another way of submitting a Sitemap file is to specify the URL in your robots.txt file. If you use this method of submitting a Sitemap file, it will be found by all search engines that support the Sitemaps protocol (although not all of them support the extensions listed above). Since you can specify the full URL of your Sitemap file in the robots.txt file, this method also allows you to store your Sitemap file on a different domain. Keep in mind that while Sitemap files submitted this way are processed on our side, they will not be automatically listed in your Webmaster Tools account. In order to receive feedback on your files, we recommend adding them manually to your account as well.
  • Using an HTTP "ping"
    If your Sitemap files are generated automatically, a convenient way to submit (and re-submit) them is to access the "ping" URL for Google Sitemaps. This URL includes the URL of your Sitemap file. For more information on the "ping" URL for your website, please see the Help Center article on Updating a Sitemap. Feel free to "ping" this URL whenever you update your Sitemap file - we'll know to pick it up and process it again. If you also have your Sitemap file registered in Webmaster Tools, we'll update the status there as well. This method is also valid if your Sitemap file is kept on a different server, but you must still verify both sites in Webmaster Tools as previously mentioned.

    Search engines that are a members of support a similar way of submitting general web Sitemap files.

We hope these simplifications make it even easier for you to send us your Sitemap files!

Tuesday, December 16, 2008

Webmaster Tools in 40 languages!

(Инструменти за уеб администратори, Eines per a administradors web de Google, Webmaster Tools, Googlen Verkkovastaavan työkalut, Εργαλεία για Webmasters, Alat WebMaster, Tīmekļa pārziņa rīki, Žiniatinkli valdytojo įrankiai, Ferramentas para o webmaster do Google, Алатке за вебмастере, Nástroje správcu webu, Orodja za spletne skrbnike, Інструменти для веб-майстра, Công cụ Quản trị Trang Web)

In our recent Webmaster Tools launch, we went live in 14 new languages, bringing our total language support count to 40! With the launch of Bulgarian, Catalan, Croatian, Filipino, Greek, Indonesian, Lithuanian, Latvian, Portuguese (Portugal), Slovak, Slovenian, Serbian, Ukrainian and Vietnamese, Webmaster Tools joins Google products such as, AdWords, Gmail and Toolbar to reach the 40 Language Initiative (Google's company-wide initiative to make sure Google products are available in the 40 languages read by more than 98% of Internet users).

Our team is very excited to reach so many of you by offering our tools in 40 languages. At the same time, both the Google Localization and Webmaster Tools teams know that there's more room for improvements in the features and quality of our service. We hope to hear your input in the comments below, especially on the linguistic quality of our new languages.

Wednesday, December 10, 2008

Message Center info through our API

Recently we mentioned some updates in the Webmaster Tools GData API: we've just launched a whole new API, the Message Center GData API, as part of the Webmaster Tools API. The Message Center is the way that Google communicates to webmasters important issues regarding their sites—for example, if there's a problem crawling your site, or if someone has requested a change in crawl rate. Until now it was only possible to access these messages through the Message Center section of Webmaster Tools; but now you can also use GData to access it as a feed. This way you don't need to continually check your messages in Webmaster Tools, you can retrieve the messages feed automatically and be informed as soon as possible of any critical issues regarding your site.
What can I do?
The Message Center GData API lets you retrieve all messages, mark the messages as read or unread, and delete messages. You can do these tasks using the provided Java client libraries, or you can create your own client code based on the protocol information.
  • Retrieve messages: The messages feed contains all the messages sent to your account. These messages have important information about your verified sites. Examples of messages include infinite spaces warnings and crawl rate change notifications.
  • Mark messages as read or unread: In order to keep track of new communications from Google, you can mark your messages as read or unread, the same way that you would manage your inbox. If you retrieve a single message, this message will be automatically marked as read.
  • Delete mesages: It's possible to delete messages using the GData API. However, be careful because if you delete a message through the API it will also be deleted in your Webmaster Tools account, as both interfaces share the same data.
How do I do it?
You can download code samples in Java for all these new features. These samples provide simple ways to use the messages feed. The following snippet shows how to retrieve the messages feed in a supported language and print all the messages:
  // Connect with the service and authenticate
  WebmasterToolsService service
new WebmasterToolsService("exampleCo-exampleApp-1");
try {
catch (AuthenticationException e) {
"Username or password invalid");

  // Retrieve messages feed
  MessagesFeed messages;
try {
    URL feedUrl;
if (USER_LANGUAGE == null) {
      feedUrl =
else {
      feedUrl =
"?hl=" + USER_LANGUAGE);
    messages = service.getFeed(feedUrl, MessagesFeed.
catch (IOException e) {
"There was a network error.");
catch (ServiceException e) {
"The service is not available.");

  // Print the messages feed
for (MessageEntry entry : messages.getEntries()) {
if (entry.getRead()) {
"   \t");
else {
    System.out.print(entry.getDate().toUiString() +

Where do I get it?
If you want to know more about GData, you may want to start by checking out the GData website. The homepage of the Webmaster Tools GData API contains a section on the messages feed, with details about the protocol. You can also download the sample Message Center client form the GData download site. It will show you how to use all the Message Center GData API features.

Sunday, December 7, 2008

Reintroducing your English Webmaster Help Google Guides

When we announced our latest round of Bionic Posters in the old Help Group, an astute webmaster noted that we had never fully introduced ourselves to the webmaster community. With our announcement of a new English Webmaster Help forum, I'm happy to (re)introduce you all to the Google Guides from around the world who will be hanging out in the English forum:

Guides in Mountain View, California (from left to right): Adam, Maile, Matt Cutts, Chris, Wysz, Matt, Chark, and Adi.

Also in Mountain View: Evan, Jessica, and Nate.

Guides in Kirkland, Washington: Susan, Riona, and Jonathan.

Guides in Zürich, Switzerland: John Mueller and Balázs.

Guides in Hyderabad, India: Koti, Reid (visiting from Mountain View), and Jayan.

Guide in Ottowa Ottawa :), Ontario: Oliver.
All of us look forward to seeing you around the forum! Our Google Guides in non-English Webmaster Help Groups introduce themselves here.

Friday, December 5, 2008

Friend Connect now available in beta to everyone

Update: The described product or service is no longer available.

If you've been looking for a way to grow traffic and make your site more interactive, check out Google Friend Connect -- now in beta and available to all webmasters. Remember that with Friend Connect, you can easily add social features to your site by just copying and pasting a few snippets of code, no programming necessary! Your visitors will be able to join your site, create or link to a profile, interact with other visitors, and invite friends to visit your site. Best of all, your visitors won't be required to create yet another username and password -- Friend Connect lets them sign in using an existing Google, Yahoo, AOL, or OpenID account. To learn more, watch the video below:

So for any of you who were interested in Friend Connect after our first announcement, and also to all of the newer readers out there, go ahead and give Friend Connect a try.

Thursday, December 4, 2008

One place for changing your site's settings

One of the many useful features of Webmaster Tools is the ability to adjust settings for your site, such as crawl rate or geographic target. We've been steadily adding settings over time and have now gotten to the point where they merit their own page. That's right, Webmaster Tools now provides a single, dedicated page where you can see and adjust all the settings for your site.

The settings that have been moved to the new Settings page are:
1. Geographic Target
2. Preferred domain control
3. Opting in to enhanced image search
4. Crawl rate control

Changing a Setting
Whenever you change a setting, you will be given an option to save or cancel the change.

Please note: The Save/Cancel option is provided on a per setting basis and hence if you change multiple settings, you'll have to click the Save button associated with each setting.

Expiration of a setting
Some of the settings are time-bounded. That is, your setting will expire after a certain time period. For example, the crawl rate setting has an expiration period of 90 days. After this period, it's automatically reset to the default setting. Whenever you visit the Settings page, you can view the date that your setting will expire underneath the setting name.

That's all there is to it!

We always like adding features and making our interface clearer based on your suggestions, so keep them coming! Please share your feedback (or ask questions) in the Webmaster Help Forum.

A new look for our Webmaster Help Group

Googlers strongly believe in dogfooding our own products. We manage our work schedules with Google Calendar, publish our blogs on Blogger, and store scads of documentation on Google Sites. So, ever since we launched our first Webmaster Help Group, we've been using Google Groups to facilitate conversations about Webmaster Tools and web search issues.
Today, however, I'm thrilled to announce that our English and Polish Help Groups are getting a makeover. And the changes are more than just skin-deep. Our new Help Forums should make it easier for you to find answers, share resources with others, and have your participation acknowledged.
You can read more about the changes on the Official Google Blog, and then check it out for yourself: English, Polish.
Q: What will happen to the old English and Polish Help Groups?
A: While our old groups are now closed to new posts, they will still be available in read-only mode in case you want to reference any of your favorite posts from the good old days. Many of the most frequently-asked questions (and answers!) have already been transferred to our new Help Forums.
Q: If I was a member of the old group, will I automatically be a member of the new forum?
A: We won't be "transferring" membership from the old groups to the new, so even if you were a member of our Google Groups forum, you'll still need to join the new forum in order to participate. Nicknames and user profiles are also managed separately, so you're welcome to recreate your Google Groups profile in our new forum, or reinvent yourself.
Q: What about the Webmaster Help Groups in other languages?
A: They'll be moving to the new Help Forum format in 2009. Specific dates will be announced in each of the groups as they get closer to their moving date.
Feel free to post any other questions about the new Help Forums in the comments below.

More control of Googlebot's crawl rate

We've upgraded the crawl rate setting in Webmaster Tools so that webmasters experiencing problems with Googlebot can now provide us more specific information. Crawl rate for your site determines the time used by Googlebot to crawl your site on each visit. Our goal is to thoroughly crawl your site (so your pages can be indexed and returned in search results!) without creating a noticeable impact on your server's bandwidth. While most webmasters are fine using the default crawl setting (i.e. no changes needed, more on that below), some webmasters may have more specific needs.

Googlebot employs sophisticated algorithms that determine how much to crawl each site it visits. For a vast majority of sites, it's probably best to choose the "Let Google determine my crawl rate" option, which is the default. However, if you're an advanced user or if you're facing bandwidth issues with your server, you can customize your crawl rate to the speed most optimal for your web server(s). The custom crawl rate option allows you to provide Googlebot insight to the maximum number of requests per second and the number of seconds between requests that you feel are best for your environment.

Googlebot determines the range of crawl rate values you'll have available in Webmaster Tools. This is based on our understanding of your server's capabilities. This range may vary from one site to another and across time based on several factors. Setting the crawl rate to a lower-than-default value may affect the coverage and freshness of your site in Google's search results. However, setting it to higher value than the default won't improve your coverage or ranking. If you do set a custom crawl rate, the new rate will be in effect for 90 days after which it resets to Google's recommended value.

You may use this setting only for root level sites and sites not hosted on a large domain like (we have special settings assigned for them). To check the crawl rate setting, sign in to Webmaster Tools and visit the Settings tab. If you have additional questions, visit the Webmaster Help Center to learn more about how Google crawls your site or post your questions in the Webmaster Help Forum.

Thursday, November 27, 2008

Date with Googlebot, Part II: HTTP status codes and If-Modified-Since

Our date with Googlebot was so wonderful, but it's hard to tell if we, the websites, said the right thing. We returned 301 permanent redirect, but should we have responded with 302 temporary redirect (so he knows we're playing hard to get)? If we sent a few new 404s, will he ever call our site again? Should we support the header "If-Modified-Since?" These questions can be confusing, just like young love. So without further ado, let's ask the expert, Googlebot, and find out how he judged our response (code).

Supporting the "If-Modified-Since" header and returning 304 can save bandwidth.

Dearest Googlebot,
  Recently, I did some spring cleaning on my site and deleted a couple of old, orphaned pages. They now return the 404 "Page not found" code. Is this ok, or have I confused you?
Frankie O'Fore

Dear Frankie,
  404s are the standard way of telling me that a page no longer exists. I won't be upset—it's normal that old pages are pruned from websites, or updated to fresher content. Most websites will show a handful of 404s in the Crawl Diagnostics over at Webmaster Tools. It's really not a big deal. As long as you have good site architecture with links to all your indexable content, I'll be happy, because it means I can find everything I need.

  But don't forget, it's not just me who comes to your website—there may be humans seeing these pages too. If you've only got a very simple '404 page not found' message, visitors who aren't as savvy could be baffled. There are lots of ways to make your 404 page more friendly; a quick one is our 404 widget over at Webmaster Tools, which will help direct people to content which does exist. For more information, you can read the blog post. Most web hosting companies, big and small, will let you customise your 404 page (and other return codes too).

Love and kisses,

Hey Googlebot,
  I was just reading your reply to Frankie above, and it raised a couple of questions.
What if I have someone linking to a page that no longer exists? How can I make sure my visitors still find what they're after? Also, what if I just move some pages around? I'd like to better organise my site, but I'm worried you'll get confused. How can I help you?
Yours hopefully,
Little Jimmy

Hello Jimmy,
   Let's pretend there are no anachronisms in your letter, and get to the meat of the matter. Firstly, let's look at links coming from other sites. Obviously, these can be a great source of traffic, and you don't want visitors presented with an unfriendly 'Page not found' message. So, you can harness the power of the mighty redirect.

   There are two types of redirect—301 and 302. Actually, there are lots more, but these are the two we'll concern ourselves with now. Just like 404, 301 and 302 are different types of responses codes you can send to users and search engine crawlers. They're both redirects, but a 301 is permanent and a 302 is temporary. A 301 redirect tells me that whatever this page used to be, now it lives somewhere else. This is perfect for when you're re-organising your site, and also helps with links from offsite. Whenever I see a 301, I'll update all references to that old page with the new one you've told me about. Isn't that easy?

   If you don't know where to begin with redirects, let me get you started. It depends on your webserver, but here are some searches that may be helpful:
You can also check your manual, or the README files that came with your server.

   As an alternative to a redirect, you can email the webmaster of the site linking to you and ask them to update their link. Not sure what sites are linking to you? Don't despair - my human co-workers have made that easy to figure out. In the "Links" portion of Webmaster Tools, you can enter a specific URL on your site to determine who's linking to it.

  My human co-workers also just released a tool which shows URLs linking to non-existent pages on your site. You can read more about that here.

Yours informationally,

Darling Googlebot,
   I have a problem—I live in a very dynamic part of the web, and I keep changing my mind about things. When you ask me questions, I never respond the same way twice—my top threads change every hour, and I get new content all the time! You seem like a straightforward guy who wants straightforward answers. How can I tell you when things change without confusing you?
Temp O'Rary

Dear Temp,
   I just told little Jimmy that 301's are the best way to tell a Googlebot about your new address, but what you're looking for is a 302.
   Once you're indexed, it's the polite way to tell your visitors that your address is still the right one, but that the content can temporarily be found elsewhere. In these situations, a 302 (or the rarer '307 Temporary Redirect') would be better. For example, orkut redirects from to, which isn't a page that humans would find particularly useful when searching for Orkut***.
It's on a different domain, for starters. So, a 302 has been used to tell me that all the content and linking properties of the URL shouldn't be updated to the target - it's just a temporary page.

  That's why when you search for orkut, you see and not that longer URL.

  Remember: simple communication is the key to any relationship.

Your friend,

*** Please note, I simplified the URL to make it easier to read. It's actually much more complex than that.

Captain Googlebot,
   I am the kind of site who likes to reinvent herself. I noticed that the links to me on my friends' sites are all to URLs I got rid of several redesigns ago! I had set up 301s to my new URLs for those pages, but after that I 301'ed the newer URLs to my next version. Now I'm afraid that if you follow their directions when you come to crawl, you'll end up following a string of 301s so long that by the end you won't come calling any more.
Ethel Binky

Dear Ethel,
   It sounds like you have set up some URLs that redirect to more redirects to... well, goodness! In small amounts, these "repeat redirects" are understandable, but it may be worth considering why you're using them in the first place. If you remove the 301s in the middle and send me straight to the final destination on all of them, you'll save the both of us a bunch of time and HTTP requests. But don't just think of us. Other people get tired of seeing that same old 'contacting.... loading ... contacting...' game in their status bar.

   Put yourself in their shoes—if your string of redirects starts to look rather long, users might fear that you have set them off into an infinite loop! Bots and humans alike can get scared by that kind of "eternal commitment." Instead, try to get rid of those chained redirects, or at least keep 'em short. Think of the humans!

Yours thoughtfully,

Dear Googlebot,
   I know you must like me—you even ask me for unmodified files, like my college thesis that hasn't changed in 10 years. It's starting to be a real hassle! Is there anything I can do to prevent your taking up my lovely bandwidth?

Janet Crinklenose

Janet, Janet, Janet,
   It sounds like you might want to learn a new phrase—'304 Not Modified'. If I've seen a URL before, I insert an 'If-Modified-Since' in my request's header. This line also includes an HTTP-formatted date string. If you don't want to send me yet another copy of that file, stand up for yourself and send back a normal HTTP header with the status '304 Not Modified'! I like information, and this qualifies too. When you do that, there's no need to send me a copy of the file—which means you don't waste your bandwidth, and I don't feel like you're palming me off with the same old stuff.

   You'll probably notice that a lot of browsers and proxies will say 'If-Modified-Since' in their headers, too. You can be well on your way to curbing that pesky bandwidth bill.

Now go out there and save some bandwidth!
Good ol' Googlebot


Googlebot has been so helpful! Now we know how to best respond to users and search engines. The next time we get together, though, it's time to sit down for a good long heart-to-heart with the guy (Date with Googlebot: Part III, is coming soon!).

UPDATE: Added a missing link. Thanks to Boris for pointing that out.

Thursday, November 13, 2008

Better targeting your indic language site

A lot has been said about how to start a multi-lingual site and how to better target content through meta tags. Our users have raised a number of interesting questions about creating websites in different languages, like the one below.

> How does one do for INDIA.
> As there are many languages spoken here.
> My Site is primarily in English, but my site targets different cities in INDIA.
> For Hyderabad - I want in Urdu & Telugu and for Chennai I want in Tamil
> for Bengaluru I want in Kannada.
> For North I want in Hindi.’

We’d like to introduce the transliteration API for Indic languages (languages spoken in India) in addition to our Ajax API for languages. With this API at your disposal, content creation is simplified because it not only helps integrating transliteration in your websites but also allows users visiting your site to type in Indic languages.

To include the transliteration API, first you need the AJAX script.

<script type="text/javascript" src=""></>

This script tag will load the google.load function, which lets you load the individual Google APIs. For loading Google Transliteration API, call to google.load looks like this:

<script type="text/javascript">
google.load("elements", "1", {
packages: "transliteration"

When it comes to targeting, don't forget to add meta tags in your local language. And for your questions, we have a new addition to our already existing communication channels like the webmaster help groups and webmaster tools (available in 26 languages!). We also have our own official Orkut webmaster community! Here users can share thoughts and discuss webmaster related issues.

Sign up for our Orkut community now and if you have any additional thoughts we'd love to hear about them.


On-Demand Sitemaps for Custom Search

Since we launched enhanced indexing with the Custom Search platform earlier this year, webmasters who submit Sitemaps to Webmaster Tools get special treatment: Custom Search recognizes the submitted Sitemaps and indexes URLs from these Sitemaps into a separate index for higher quality Custom Search results. We analyze your Custom Search Engines (CSEs), pick up the appropriate Sitemaps, and figure out which URLs are relevant for your engines for enhanced indexing. You get the dual benefit of better discovery for and more comprehensive coverage in your own CSEs.

Today, we're taking another step towards improving your experience with Google webmaster services with the launch of On-Demand Indexing in Custom Search. With On-Demand Indexing, you can now tell us about the pages on your websites that are new, or that are important and have changed, and Custom Search will instantly schedule them for crawl, and index and serve them in your CSEs usually within 24 hours, often much faster.

How do you tell us about these URLs? You guessed it... provide a Sitemap to Webmaster Tools, like you always do, and tell Custom Search about it. Just go to the CSE control panel, click on the Indexing tab, select your On-Demand Sitemap, and hit the "Index Now" button. You can tell us which of these URLs are most important to you via the priority and lastmod attributes that you provide in your Sitemap. Each CSE has a number of pages allocated within the On-Demand Index, and with these attributes, you can us which are most important for indexing. If you need greater allocation in the On-Demand index, as well as more customization controls, Google Site Search provides a range of options.

Some important points to remember:
  1. You only need to submit your Sitemaps once in Webmaster Tools. Custom Search will automatically list the Sitemaps submitted via Webmaster Tools and you can decide which Sitemap to select for On-Demand Indexing.
  2. Your Sitemap needs to be for a website verified in Webmaster Tools, so that we can verify ownership of the right URLs.
  3. In order for us to index these additional pages, our crawlers must be able to crawl them. You can use "Webmaster Tools > Crawl Errors > URLs restricted by robots.txt" or check your robots.txt file to ensure that you're not blocking us from crawling these pages.
  4. Submitting pages for On-Demand Indexing will not make them appear any faster in the main Google index, or impact ranking on
We hope you'll use this feature to inform us regularly of the most important changes on your sites, so we can respond quickly and get those pages indexed in your CSE. As always, we're always listening for your feedback on Custom Search.

Wednesday, November 12, 2008

Google's SEO Starter Guide

Note: The SEO Starter Guide has since been updated.

Webmasters often ask us at conferences or in the Webmaster Help Group, "What are some simple ways that I can improve my website's performance in Google?" There are lots of possible answers to this question, and a wealth of search engine optimization information on the web, so much that it can be intimidating for newer webmasters or those unfamiliar with the topic. We thought it'd be useful to create a compact guide that lists some best practices that teams within Google and external webmasters alike can follow that could improve their sites' crawlability and indexing.

Our Search Engine Optimization Starter Guide covers around a dozen common areas that webmasters might consider optimizing. We felt that these areas (like improving title and description meta tags, URL structure, site navigation, content creation, anchor text, and more) would apply to webmasters of all experience levels and sites of all sizes and types. Throughout the guide, we also worked in many illustrations, pitfalls to avoid, and links to other resources that help expand our explanation of the topics. We plan on updating the guide at regular intervals with new optimization suggestions and to keep the technical advice current.

So, the next time we get the question, "I'm new to SEO, how do I improve my site?", we can say, "Well, here's a list of best practices that we use inside Google that you might want to check out."

Update on July 22, 2009: The SEO Starter Guide is now available in 40 languages!

Friday, October 31, 2008

Spookier than malware


lion king
...and infinitely more fun: webmasters and their pets incognito! Happy Halloween, everyone! If you see any costumes that would pass the SafeSearch filter :), feel like sharing a gripe or telling a good story, please join the chat!

Take care, and don't forget to brush your teeth.
 Yours scarily,
  The Webmaster Central Team

Our glasses-wearing, no vampire-teeth vampire (Ryan), zoombie Mur, Holiday Fail (Tiffany Lane), Colbert Hipster (Dan Vanderkam), Rick Astley Cutts, Homeboy Ben D'Angelo, Me -- pinker & poofier, Investment Bank CEO Shyam Jayaraman (though you can't see the golden parachute in his backpack)

Chark as Juno, Wysz as Beah Burger (our co-worker), Adi and Matt Dougherty as yellow ninja, red ninja!

Heroes come in all shapes and sizes...

Powdered toast man, Mike Leotta

Adam Lasnik as, let me see if I get this right, a "secret service agent masquerading as a backstage tech" :)

Wednesday, October 29, 2008

Reflections on the "Tricks and Treats" webmaster event

What featured over 750 webmasters and a large number of Googlers from around the world, hundreds of questions, and over one hundred answers over the course of nearly two hours?  If you guessed "the Tricks and Treats webmaster event from this earlier this month!" well, you're either absolutely brilliant, you read the title of this post, or both!

How did it go?
It was an exhilarating, exhausting, and educational event, if we may say so ourselves, even though there were a few snafus.  We're aware that the sound quality wasn't great for some folks, and we've also appreciated quite-helpful constructive criticisms in this feedback thread.  Last but not least, we are bummed to admit that someone (whose name starts with 'A' and ends with 'M') uncharacteristically forgot to hit the record button (really!), so there's unfortunately no audio recording to share :-(.

But on more positive notes, we're delighted that so many of you enjoyed our presentations (embedded below), our many answers, and even some of our bad jokes (mercifully not to be repeated).

What next?
Well, for starters, all of us Webmaster Central Googlers will be spending quite some time taking in your feedback.  Some of you have requested sessions exclusively covering particular (pre-announced) topics or tailored to specific experience levels, and we've also heard from many webmasters outside of the U.S. who would love online events in other languages and at more convenient times.  No promises, but you can bet we're eager to please!  Stay tuned on this blog (and, as a hint and hallo to our German-speaking webmasters, do make sure to follow our German webmaster blog  ;-).  

And finally, a big thank you!
A heartfelt thank you to my fellow Googlers, many of whom got up at the crack of dawn to get to the office early for the chat and previous day's runthrough or stayed at work late in Europe.  But more importantly, major props to all of you (from New Delhi, New York, New Zealand and older places) who asked great questions and hung out with us online for up to two hours.  You webmasters are the reason we love coming to work each day, and we look forward to our next chat!

*  *  *

The presentations...
We had presentations from John, Jonathan, Maile, and Wysz.  Presentations from the first three are embedded below (Wysz didn't have a written presentation this time).

John's slides on "Frightening Webmastering Myths"

Jonathan's slides on "Using the Not Found errors report in Webmaster Tools"

Maile's slides on "Where We're Coming From"

Edited on Wednesday, October 29 at 6:00pm to update number of participants

Friday, October 24, 2008

Malware? We don't need no stinking malware!

(Cross-posted from the Google Online Security Blog.)

"This site may harm your computer"
You may have seen those words in Google search results — but what do they mean? If you click the search result link you get another warning page instead of the website you were expecting. But if the web page was your grandmother's baking blog, you're still confused. Surely your grandmother hasn't been secretly honing her l33t computer hacking skills at night school. Google must have made a mistake and your grandmother's web page is just fine...

I work with the team that helps put the warning in Google's search results, so let me try to explain. The good news is that your grandmother is still kind and loves turtles. She isn't trying to start a botnet or steal credit card numbers. The bad news is that her website or the server that it runs on probably has a security vulnerability, most likely from some out-of-date software. That vulnerability has been exploited and malicious code has been added to your grandmother's website. It's most likely an invisible script or iframe that pulls content from another website that tries to attack any computer that views the page. If the attack succeeds, then viruses, spyware, key loggers, botnets, and other nasty stuff will get installed.

If you see the warning on a site in Google's search results, it's a good idea to pay attention to it. Google has automatic scanners that are constantly looking for these sorts of web pages. I help build the scanners and continue to be surprised by how accurate they are. There is almost certainly something wrong with the website even if it is run by someone you trust. The automatic scanners make unbiased decisions based on the malicious content of the pages, not the reputation of the webmaster.

Servers are just like your home computer and need constant updating. There are lots of tools that make building a website easy, but each one adds some risk of being exploited. Even if you're diligent and keep all your website components updated, your web host may not be. They control your website's server and may not have installed the most recent OS patches. And it's not just innocent grandmothers that this happens to. There have been warnings on the websites of banks, sports teams, and corporate and government websites.

Uh-oh... I need help!
Now that we understand what the malware label means in search results, what do you do if you're a webmaster and Google's scanners have found malware on your site?

There are some resources to help clean things up. The Google Webmaster Central blog has some tips and a quick security checklist for webmasters. has great information, and their forums have a number of helpful and knowledgeable volunteers who may be able to help (sometimes I'm one of them). You can also use the Google SafeBrowsing diagnostics page for your site (<site-name-here>) to see specific information about what Google's automatic scanners have found. If your site has been flagged, Google's Webmaster Tools lists some of the URLs that were scanned and found to be infected.

Once you've cleaned up your website, use Google's Webmaster Tools to request a malware review. The automatic systems will rescan your website and the warning will be removed if the malware is gone.

Advance warning
I often hear webmasters asking Google for advance warning before a malware label is put on their website. When the label is applied, Google usually emails the website owners and then posts a warning in Google's Webmaster Tools. But no warning is given ahead of time - before the label is applied - so a webmaster can't quickly clean up the site before a warning is applied.

But, look at the situation from the user's point of view. As a user, I'd be pretty annoyed if Google sent me to a site it knew was dangerous. Even a short delay would expose some users to that risk, and it doesn't seem justified. I know it's frustrating for a webmaster to see a malware label on their website. But, ultimately, protecting users against malware makes the internet a safer place and everyone benefits, both webmasters and users.

Google's Webmaster Tools has started a test to provide warnings to webmasters that their server software may be vulnerable. Responding to that warning and updating server software can prevent your website from being compromised with malware. The best way to avoid a malware label is to never have any malware on the site!

You can request a review via Google's Webmaster Tools and you can see the status of the review there. If you think the review is taking too long, make sure to check the status. Finding all the malware on a site is difficult and the automated scanners are far more accurate than humans. The scanners may have found something you've missed and the review may have failed. If your site has a malware label, Google's Webmaster Tools will also list some sample URLs that have problems. This is not a full list of all of the problem URLs (because that's often very, very long), but it should get you started.

Finally, don't confuse a malware review with a request for reconsideration. If Google's automated scanners find malware on your website, the site will usually not be removed from search results. There is also a different process that removes spammy websites from Google search results. If that's happened and you disagree with Google, you should submit a reconsideration request. But if your site has a malware label, a reconsideration request won't do any good — for malware you need to file a malware review from the Overview page.

How long will a review take?
Webmasters are eager to have a Google malware label removed from their site and often ask how long a review of the site will take. Both the original scanning and the review process are fully automated. The systems analyze large portions of the internet, which is big place, so the review may not happen immediately. Ideally, the label will be removed within a few hours. At its longest, the process should take a day or so.

Tuesday, October 21, 2008

Webmaster chat event: Vote early and often!

No matter where in the world you are, you can vote right now on webmaster-oriented questions by registering for our free Webmaster chat  ("Tricks and Treats") which is scheduled for tomorrow at 9am PDT (5pm GMT).  Even better: you can suggest your own questions that you'd like Webmaster Central Googlers to answer.

We're using the new Google Moderator tool, so posting questions and voting on your favorites is fun and easy; you'll receive an e-mail with a link to the webmaster chat questions right after you register.  Click on the check mark next to questions you find particularly interesting and important. Click on the X next to questions that seem less relevant or useful.  From your votes, Google Moderator will surface the best questions, helping us spend more time in the chat on issues you really care about.

Feel free to review our post from yesterday for more details on this event.

See you there!

P.S. - Speaking of voting:  If you're an American citizen, we hope you're also participating in the upcoming presidential election! Our friends in Google Maps have even prepared a handy lookup tool to help you find your voting place -- check it out!