So you’re not sure if your site is really getting indexed like it ought to. You’ve got a couple of options here. You can:
- Hope for the best
- Do something about it!
Experience has taught me that relying on hope to generate results is about as effective as trying to get in shape without lifting a finger.
And since we don’t have a magic pill here, I think it’s best that we try something we know will help our site get indexed by our friends at Google, Yahoo!, and MSN Search.
Today, you’re going to strike up an ongoing conversation with the search engines by adding an XML sitemap to your soon-to-be-awesome Web site.
How do you generate an XML Sitemap?
Unless you have some experience writing XML, then generating a sitemap to spec would prove to be a real challenge. And even if you were able to pull this off, you’d still have to figure out a way to automate sitemap updates.
Drag.
As if there weren’t already enough reasons to switch to WordPress, then hopefully you’ll find this one compelling enough to get over that hurdle.
There are WordPress plugins for damn near everything, and if you’re talking about standardized Web site features (such as a sitemap), then I can pretty much guarantee that there’s a plugin out there to solve your problem.
That said, it should come as no surprise that there is an extremely handy plugin available that takes care of our XML sitemap issue. It’s called the Google Sitemap Generator, and you’ll want to download the latest version.
Implementing the Google Sitemap Plugin for WordPress
I’m going to take a leap of faith here and assume you’re familiar with the process of installing and activating WordPress plugins, but if you need help with those details, please check out WordPress hacks 2–5 to see how that process generally works.
Once you’ve activated the plugin, you’re basically good to go. The default settings will work fine for just about every site, but if you’re the type who likes to tinker and tweak every little detail, then the Google Sitemap Plugin options page ought to make you positively giddy.
How the Plugin Works
The Google Sitemap Plugin automagically pings Google, Yahoo!, and MSN Search at regular intervals, notifying them of new content and/or changes to your site. You literally don’t have to do anything in order to make it “work.”
Personally, though, I recommend that you take one extra step and actually link to the sitemap from your home page. As you can see here, I stuck the link in my footer, and as a result, it appears on every page of the site.
While I don’t know that there’s any direct benefit from doing so, I link to my XML sitemap for two reasons:
- I want to showcase the fact that I support this standardized format.
- As a reference point (albeit a small one) for this industry, I want to make public all of the “best practices” that I employ so that I may help others do the same.
And I guess ultimately, I do want you to know that I’m talking to Google on a regular basis. Maybe one day we’ll do lunch.
This is the first in a series of posts covering SEO “best practices” called SEO for Everybody.
There are lots of simple tweaks that average, everyday users can make to their Web sites that will drastically improve both overall accessibility and search engine performance. The goal of SEO for Everybody is to help guide those users through these basic processes, thus resulting in a more accessible, utopian universe.
Or something like that
41 Comments ↓
Chris, I noticed a couple of days ago that you put this xml sitemap into your footer, but the xhtml did not seem to work - I see it does now. Will you also talk about how you generate that? The xml is really easy with this plugin, I think it’s simply great. And thanx for the first “tutorial”!
Google Sitemaps are indeed an essential tool for bloggers. However, I fail to see the necessity (or even the potential benefit) of linking to your XML sitemap. The XML is meant for Google to read, and most readers probably aren’t interested in parsing it by eye.
I can’t imagine the typical internet user finding anything but confusion with an XML link. I agree that meeting standards should be noted and credited- but having valid XHTML and CSS is a little different from installing a sitemap plugin.
Great Post Chris,
I was wondering if you wouldn’t mind sharing some of the great plug-ins your currently using for Wordpress. I’m just starting to get my own blog put together and see the value in some great addons.
Thanks,
Zoltan — That’s the next post
Alex — As I indicated in the post, I don’t see any real benefit to including that link on the home page. Like I said, though, I just want to put it there so people will see that I am abiding by what I consider to be best practices for SEO.
Steve — A colophon for this site is in the works, and it will include a list of the plugins I use and the purpose that each one serves. When I finally finish it, I’ll cover it in an aside.
Chris,
Very informative post and easy to implement. Also, are meta tags important for SEO. Some say no but put them in anyway. Whats your take?
Mike,
Without question,
metatags seem to be falling out of favor, at least in a lot of SEO conversations that I’ve witnessed over the past few months.Despite that, however, I still think it is a good idea to generate dynamic
metatags on each page if possible. I definitely think that Google is assigning less and less importance to this aspect of Web sites, but since it’s still out there, you may as well take advantage of it.I’m all about gaining as much leverage as possible, ya know?
I’ve noticed your not doing any google ads on your newest articles. Why?
Very helpful article Chris. I’ve been looking for a sitemap generator for a while, knowing I now have one that has been tried and tested makes me feel much more confident about it.
Lets hope Google likes my site.
MSN and Yahoo announced they were going to jump on the sitemap bandwagon.
I guess that means my parents will actually be able to find my site now
Sharon,
I’ve got about a month of ad placement testing under my belt now, and I’ve come to the conclusion that there is only one location where ads are ever going to be effective on this site. Where, you ask?
Flush left in the post copy, directly underneath the title, and the text of the article must wrap around the ads. Furthermore, I’ve achieved the highest CTR by using the medium rectangle, which “weighs” in at 300 x 250 px.
That said, I am using the SEO for Everybody image to brand this series of posts, and the image really cannot occupy its current position if I place the ads where they’ll be most effective.
I haven’t bothered with an alternative yet, but I may tinker with that today. On one hand, I hate to leave money on the table. On the other hand, though, I hate to mess with the overall presentational qualities of my posts. I’m sure there’s a happy medium in there somewhere; I just have to find it
[...] After asking Chris why we need both sitemap generators, 1 x xml sitemap and 1 x xhtml sitemap, he replied: [...]
Hi Chris.
I have a question. My blog is a directory of my portoflio site. So, if I place the sitemap in the directory will it also include the original portfolio pages?
My portfolio is http://www.davidairey.com whereas my blog is http://www.davidairey.com/blog (where the wordpress sitemap is added).
Many thanks.
Ah, excuse me, I just checked the plugin options in wordpress and that has helped.
I’ll leave the sitemap in the /blog directory and add the few pages ‘above’ it that form my portfolio.
David,
You’re on the right track. The XHTML Sitemap for WordPress is only capable of picking up pages (and posts) that have been created within the WordPress architecture.
Anything else would have to be hard-coded into the
sitemap.phpfile.Ah ok, I notice that the XML sitemap has the ability to pick up pages ‘above’ the homepage directory however, by simply ‘adding a new page’ in the wordpress plugin options.
Here’s a question for you Chris. I wonder if you’ve come across it:
Is there an ideal relationship between a particular ‘change frequency’ for sitemaps and the ’sitemap priority’?
For instance, say I set a page on the sitemap to change monthly, is there an optimum priority setting?
If so, I think it could be useful to publish a table along the lines of:
Frequency = daily / priority = 1.0
Frequency = yearly / priority = 0.1
To be honest I was guessing when I filled that one in.
David,
I really have no idea about that. When I first added the sitemap here, I briefly looked into those options. Ultimately, I decided that it wasn’t going to make a whole heck of a lot of difference to Google either way.
Google offers Webmaster tools that allow you to check how often your site is being indexed. I’m currently watching this, and I’ll be writing a post about it in the not-too-distant future.
Potentially, there may be some correlation between your sitemap settings and how often your site gets crawled, but we’ll have to wait and see.
Hi Guys!
I am trying to use this. My host is same - MidPhase. And I cannot use this plugin!
Gives me the following error
# December 11, 2006 9:03 pm: Could not write into /home/eashnak8/public_html/alpesh/blog/sitemap.xml.gz: fopen(/home/eashnak8/public_html/alpesh/blog/sitemap.xml.gz) [function.fopen]: failed to open stream: Permission denied
# December 11, 2006 9:03 pm: Could not write into /home/eashnak8/public_html/alpesh/blog/sitemap.xml: fopen(/home/eashnak8/public_html/alpesh/blog/sitemap.xml) [function.fopen]: failed to open stream: Permission denied
What am I missing here?
Cheers!
Alpesh,
It looks as though you need to change the permissions on your
/blogto allow it to be writeable by the server.If you’re familiar with that process, you can do it yourself. It’s fairly simple, and there should be lots of tutorials online that explain how it’s done.
Otherwise, you can call MidPhase, and I’m sure one of their techs will be happy to help you out.
[...] Naturally, it’s not by mere chance that this has suddenly happened. I’ve implemented several SEO1 techniques based on those shared by Chris Pearson over at pearsonsonified.com. Thanks for the awesome tips Chris! Search Engine Optimization ↩Share This [...]
Chris,
One thing I missed in your post is that you have to tell Google and Yahoo that you have a sitemap.xml ready for them…
For Google you can do this by logging with your Gmail account name into webmastercentral and then webmaster tools (find it via Google’s About).
For Yahoo you have to login and use there Site Explorer.
Microsoft is still eveluating how do wil do this.
But it is not picked up if you just publish it…
http://blog.design-cars.com/sitemapxml-standard-like-robotstxt-for-google-live-and-yahoo-and-others/
As for the Metatags, there is a plugin as well
http://wordpress.uberdose.com/wordpress/another-wordpress-meta-plugin.html
I always fill the description tag, and up to five keywords I used in the post content for the keywords tag….
There are more search engines then Google and the use the Metatag descriptions, as sometimes Google does as well.
P.s. There is more information te be gathered from Webmaster tools.
Maybe I will write a post on that on blog.design-cars.com.
Hope this helps on your SEO as well
Hi Chris .. I’ve finally had time to add the google sitemap plugin to my sites without a sitemap .. and one of them is kicking up tons of errors..
Paths don’t match .. Do you know offhand of an easy way around this? It’s on the site linked above.
t.i.a.
Hello Hart
If you check in to Webmaster central and then pick your site with the problem.
In the diagnostics tab there is a tools section with the item “preferred domain” you could tell Google how to handle the http://www. extensions. In you case this shoul be the upper choice.
My favorite solution is to ask you provider (or do it yourself from your control panel) to set a redirect the http://hart-network.com/ to http://www.hart-network.com/
Most people use the http://www. before the domain name.
And if the don’t, Some browsers will place it before the domain name automaticly.
Hope this helps to solve your problem.
Hart,
Have you checked out the discussion on the sitemap plugin site, or perhaps tried Googling “WordPress sitemap plugin?”
I’m not really sure why you would be experiencing that particular problem, but hopefully someone else has fought the same battle and figured out an answer!
That was my next step, although .. I figure it wouldn’t hurt to ask
just in case there was an easy solution. // thanks!
Pathos .. that was quite a few helpful tips! But, why do you think it’s better to include the www?
I believe I know where to set the redirect in my cPanel of all my accounts .. I was of the thought that NO www would be the way to go .. although, I see in the URL here at pearsonified that www is included.. and will lean towards that for google purposes. Thanks.
To Hart,
Glad I could help.
the www is used because vistitors are mostly “trained” that way…
Chris,
I’ve been using this plugin on 2.0.5, but have noticed it doesn’t produce correct links for child pages.
In other words, create a page with a parent, then run the sitemap, and the child page appears with the incorrect link. This causes Google to report it wrongly as well.
Is this just my setup? Or do you see problems with child pages as well?
Glenn,
I’ve never created (or tested) child pages, so I can’t answer that reliably!
Despite that, I’d venture to say that the plugin probably doesn’t handle child pages properly, as I can see that being a test condition that is rarely encountered (or accounted for, if you will).
Chris, I’ve tested and also reviewed the code - its clear parent pages were reported incorrectly.
I’ve posted a patch for this error (tested on WP2.0.5) on our blog.
Hello Glenn,
I did some testing on this mather, and I have no problems.
Sitemap.xml and parent and child pages, all fine by Google.
But then I run the stable version 2.7.1. of the plugin, and you are running version 3.0b4.
Now the b in this number stands for Beta….
I myself, am always more happier running the stable version for my live websites.
That is a good point. I’m testing on WP 2.0.5 and using the 3.0b4 version and noticing the error. The 2.71 version uses a different approach to getting the URI, and gets the right result for child pages using WP2. I’ll update my post accordingly.
V2.71 was last changed in July 05 in the plugin change history, The beta version is a continuation of 2.7, and has updates all through 2006. This includes excluding future posts, excluding password protected posts and performance/memory fixes (possibly getting the post name via the select - a key difference between 2.71 and the beta - was part of the performance improvement).
Google moved the goalposts on ‘beta’. A while ago I would be 100% with you on not running beta for live sites, but these days avoiding beta software altogether would mean missing out on some good software.
[...] Update: this error only applies in 3.0b4 (the beta) - thanks to Pathos for reporting over at Pearsonified that v2.71 (the stable version) works correctly for child pages. [...]
I tried 3.0b6 of the plugin but the sitemap is produces is not valid. When the sitemap is submitted to Google, it gives the following error: “Unsupported file format
Your Sitemap does not appear to be in a supported format. Please ensure it meets our Sitemap guidelines and resubmit”.
Chris, hi,
I’m pretty new to all this, so please excuse these 2 newbie questions…!
1) Is there any security benefit to putting a writeable sitemap.xml file and a sitemap.xml.gz file into the /blog directory rather than making the whole directory writeable?
2) I was confused by an earlier post that said you had to notify Google and Yahoo if you place the sitemap files, as it seems to contradict the documentation on the plugin which says that Google and Y are pinged automatically. Can you confirm whether or not I need to notify them if I place these files..?
Many thanks!
Martin.
@Martin
yes, the security should only be on those files, otherwise some Bad,bad guys could take over your blog by using indirect rewrites off for example your index.php…
As for the Notification of Google and Yahoo, you can do this by login in the Google webmaster tools and Yahoo explorer.
Its easier to tell them, so you can track if every thing works out fine.
Otherwise look at http://blog.design-cars.com/sitemapxml-and-autodiscovery-by-google-ask-yahoo-and-live-via-robotstxt/ for an explanation of the autodiscovery of sitemap.xml
Pathos, thanks!
Again - please excuse my dumb questions but this is my first time doing this…
My blog is in a sub-directory (wealthydragon.com/blog) but there is no robots.txt file there. Am I correct to assume that there is only 1 robots.txt file in a site - i.e. in wealthydragon.com there will be only 1 robots.txt file..?
If so, I guess this will be in my root directory and therefore I need to add the full url of my blog’s sitemap.xml file into the robots.txt file in my root directory… Is that correct..?
Again - apologies for my dumb questions and many thanks for your patience and help!
Martin.
@Martin
You can create or copy (and edit) the one from your main site into your Blog directory without any problems, the Searchbots will find them.
You can have different xml files for each of the sites.
Terrific,
Simple to install, easy to configure
My WP is in the main directory so I made the 2 files necessary and gave them write permission and everyhing worked fine,
Thanks
Richard Schulz
“Americas Most Opinionated Man”
Google’s Webmasters Tools are an essential part of any webmasters arsenal, yet trying to literally talk to them is nigh on impossible.
After a mishap with some advertising accounts, trying to rectify the problem with contacting them took 6 days!
There are no phone numbers for Google’s Adwords
Criss i always talk to google praying for a higher pagerank on our free site Barca4ever.mysites.nl
Maybe its simple for all the people here but for us
using a low budget and no seo experience a disaster!
Can you help us or its better to praying
harder to Google for a better pagerank?
Found this post from a link on the new thesis theme. Can’t believe I missed it earlier. I’ll be installing that plug in soon.
Hoot and/or Holler ↓