|
Some crawler-based search engines make it easy to confirm that your web page
is in their index. With others, it can be more difficult. Below are the best
ways to find your web pages in the major crawler-based search engines.
Please note that these commands can also be useful for web searchers who
wish to refine their queries, as explained more in the Site
Search section of the Power Searching For Anyone page.
The Search Engine Alliances page explains where some search engines get
or give their listings to others. Also, Search Engine Watch members have
access pages that explain in depth the different data sources that each
major search engine users and how they display information.
AltaVista
AltaVista has commands that can be used to easily narrow your search to a
single URL or to pages within a particular web site. These commands can also
be combined with query terms by those who wish to refine their search
results.
URL Search
To find a single page listed in AltaVista's crawler-based index, you can
use the "url:" command. Simply preface the URL you wish to locate
with this command, such as:
url:http://searchenginewatch.com/webmasters/meta.html
If the URL is in the index, it will be displayed. You can also use this
command to find pages within a particular section of a web site. For example,
this:
url:http://searchenginewatch.com/webmasters/
would list all the pages from within the /webmasters/ area of Search
Engine Watch. This can be a useful way to find all the pages from your web
site, if it resides within someone else's domain.
Site Search
To locate all the URLs listed from a particular web site, use the
"host:" command, such as:
host:searchenginewatch.com
Use only the actual domain name. Omit the http:// prefix. Also, be aware
that using the www prefix can make a difference. For instance, the query
below:
host:www.searchenginewatch.com
would bring back only the pages AltaVista has found from Search Engine
Watch with the www prefix. However, Search Engine Watch can also be reached
without the www prefix. In fact, this is the more common way that people
come to Search Engine Watch. Consequently, AltaVista has actually indexed
many more pages from the site without the www prefix. To see these pages,
the first example shown would have to be used.
Excite
You won't find Excite's URL or site search commands documented in its
help files, but they work nonetheless.
URL Search
To find a single page listed in Excite's crawler-based index, you can use
the "url:" command.
url:www.searchenginewatch.com/webmasters/meta.html
The command works the same as with AltaVista, which means you can also
use it to find pages within a particular section of a web site. Be sure to
omit the http:// prefix. Using a www prefix will make a difference. If you
don't find your page when using it, also try searching without it.
Site Search
To locate all the URLs listed from a particular web site, use the
"site:" command, such as:
site:searchenginewatch.com
The site command works exactly the same as AltaVista's host command with
one exception: it can only be used to locate web pages. You cannot combine
it with search terms in order to narrow your results. The URL
Search section of the Power Searching For Anyone page explains this
more.
AllTheWeb.com/FAST Search
At FAST Search, commands can be used to find a single URL or multiple web
pages from a particular site, as explained below:
URL Search
To find a single page listed in FAST's crawler-based index, you can use
the "url.all:" command:
url.all:searchenginewatch.com/webmasters/meta.html
This command will also work to bring up a single URL that is listed in
the FAST-powered results used by Lycos.
Site Search
To locate all the URLs listed from a particular web site, use the "url.host:"
command, such as
url.host:searchenginewatch.com
Use only the actual domain name. Omit the http:// prefix. Also, be aware
that using the www prefix can make a difference, as described with
AltaVista.
Google
At Google, commands can be used to find a single URL or multiple web
pages from a particular site, as explained below:
URL Search
To find a single page listed in Google's crawler-based index, you can use
the "allinurl:" command, such as:
allinurl:searchenginewatch.com/webmasters/meta.html
The allinurl command works the same as with AltaVista, which means you
can also use it to find pages within a particular section of a web site. Be
sure to omit the http:// prefix.
Please note that if you are trying to find web pages with both words in
the URL and in the document itself, you'll need to use the special "inurl"
command. This is explained more in the URL
Search section of the Power Searching For Anyone page.
Site Search
To locate all the URLs listed from a particular web site, use the
"site:" command in combination with a word or words that you know
appear on all the pages. For example:
site:searchenginewatch.com
searchenginewatch
would bring up all (or nearly all) of the pages Google lists from Search
Engine Watch, because all the pages should have the word "searchenginewatch"
on them as part of the footer text.
You must use the site command in combination with a search term. It will
not work, otherwise.
Inktomi
Inktomi powers some of the results used by a variety of different search
engines. Below is how to locate a single or multiple URLs within Inktomi
powered-listings.
URL Search
To find a particular URL listed in Inktomi's crawler-based index, you can
use the "originurl:" command. Simply preface the URL you wish to
locate with this command, such as:
originurl:http://searchenginewatch.com/webmasters/meta.html
If the URL is in the index, it will be displayed. This command has been
tested to work on the following Inktomi-powered services:
The originurl command will bring up an individual URL listed in the
Inktomi-powered results of these services. It does not work at iWon,
LookSmart or MSN Search.
Keep in mind that not all Inktomi partners tap into the entire Inktomi
database. That's one reason why you may find a URL at one service but not at
another.
Site Search
To locate all the URLs listed from a particular web site, use the
"domain:" command, such as:
domain:searchenginewatch.com
Use only the actual domain name. Omit the http:// prefix. As
explained above
for AltaVista, using the www prefix can also make a difference.
Unfortunately, the domain command works inconsistently at different
Inktomi-powered services. Here's a rundown:
HotBot: Use the command. If your have any listings in the Open
Directory, these will be shown first. You'll know these are Open Directory
listings because they will have a "More like this" link underneath
them. By pass these and find a listing for your site that instead has a
"See results from this site only" link. Select that link, and you'll
see all the pages listed in Inktomi from your web site.
iWon & LookSmart: Using the command will list all pages from a
web site.
At the following services, the command fails to operate because "clustering"
(as explained on the Search
Assistance Features page) prevents you from seeing more than a few pages
from your site.
- AOL Search
- GoTo
- MSN Search
Directory Search
Inktomi has a special command that lets you find pages within a specific
areaof a web site. This is the "originurlpath:" command, and you
use it in combination with the domain command, such as:
domain:searchenginewatch.com
originurlpath:webmasters
This would find pages from within the /webmasters/ area of Search Engine
Watch. In other words, everything within this area would be listed:
http://searchenginewatch.com/reports/ekgs/
Directories: Yahoo, LookSmart & The
Open Directory
Directories are search engines that are powered by human beings, rather
than by crawling the web. Because humans are involved, directories tend to
list only a few pages per web site. This means that you probably won't be
needing to make use of special site or URL commands to locate your listings.
In fact, of the three major directories, only Yahoo has any specific command
like this. At Yahoo, you can use the "u:" command to locate
specific URLs, like this:
u:searchenginewatch.com
That would bring up any pages from Yahoo's human-compiled listings that
contain "searchenginewatch.com" within the URL.
At the web's two other major directories, LookSmart and the Open
Directory, you'll find that searching for your domain or a portion of your
domain should bring up many or all of your listings.
For example, by entering "searchenginewatch.com" or "searchenginewatch,"
I would be able to find most of my human-compiled listings in both places.
LookSmart also provides a detailed guide to locating your URL within its
service and the listings it provides to partners:
How to Find Your Listing in the LookSmart Network
http://submit.looksmart.com/info.jhtml?page=find
Outsourcers
All the search engines and directories mentioned above produce their own
listings. However, there remain some major search engines that simply
outsource to produce the results they provide. For example, MSN Search uses
information from both LookSmart and Inktomi. Trying to check your listings
at such a place is difficult, because two or more data sets are involved.
Below is a guide to what happens as such places
AOL Search
Use tips described above for the Open Directory to find your
Open Directory listings and for Inktomi to find your Inktomi listings.
HotBot
Using the Inktomi site search command described above will brings up both
your Open Directory and Inktomi listings. There is no command to bring up
all your pages included in HotBot's Direct Hit powered "Top Ten"
listings.
Lycos
Use tips described above for FAST to find your FAST listings. There is no
way to find all your listings from the Open Directory or Direct Hit
information the service uses.
MSN Search
Use tips described above for Inktomi to find your Inktomi listings. There
is no way to find all your listings from LookSmart.
Netscape
Use tips described above for the Open Directory to find your
Open Directory listings and for Google to find your Google listings.
Other Resources
There are services that can check search engines for your URL
automatically, including checking on how they appear in relation to
particular keyword phrases. These are listed on the URL
Checking Services page.
Also, this page gives you just a taste of some of the powerful searches
that can be done with search engines. See the Power
Searching page for an at-a-glance guide to other types of searches.
|