Can you use web search engines as front ends to your database?

  • Thread starter Thread starter disaia
  • Start date Start date
D

disaia

2 problems:

Example: If a person types in a part number into Yahoo:

1. Is there a way for Yahoo to list your web site as one of the
results.

2. If the user clicks on your link, can your web application know the
part number the user typed into the Yahoo search box. I would like to
use that part number to query our database and present a dynamic web
page with the details of that part number.


I know one way to solve this is to create a web page for every part
number, but since there are over 10 million part numbers it wouldn't be
the best solution. I need a way to dynamically generate that web page
(hopefully with VB.NET, my prefered language).
Thanks,
Dominic Isaia
(e-mail address removed)
 
Search engines crawl each web site, often months after they "discover" their
existence (which they do when you tell them about you, by listing on one of
the listing services). The Microsoft Small Business Center has a nice
registration service that informs search engines of the existence of your
site (see http://www.microsoft.com/smallbusiness/bc/default.mspx )

So, if you want a search engine to return a hit when a part number is
entered, you need to make sure that that:

a) your web site has a complete, navigable, tree containing your entire
catalog, with part numbers and complete descriptions of the product on every
page. Crawlers will not run your Search function, nor will they run
Javascript, nor will they follow a link that varies from another link only
by a parameter, so it has to be straight-out-navigation. This usually
requires you to implement an HTTPModule
(http://www.devx.com/vb2themax/Article/19901) so that you can embed the
parameters directly into the name of the "resource".

For example, browse over to Amazon. Drill down from the menus. You will
find that, technically, you can get to EVERY product in the Amazon catalog
without using Javascript and by simply linking from one page to another.
After you've found a book, look at the URL. It will be a long custom string
(they invented their own page engine called obidos). Select all parts of
the string BEFORE the ?. (That is the only part that a crawler will keep).
Now, copy that into another browser window. You will find the exact same
book. In other words, the crawler needs to find the part by urls that look
like this:
http://site.example.com/catalog/part42134321
and not a url that looks like this
http://site.example.com/showpage.aspx?part=42134321

b) you should find a way to increase your link visibility, by getting as
many other sites to link to yours as possible. Crawlers find you that way
too. I recommend that you look at partnerships and relationships with other
sites, so that they will link you frequently. Once again, the links should
be as specific as possible and must not rely on parameters to find the part.

c) pray that your part numbers are relatively unique compared to part
numbers from other products or from other companies.

Good Luck,

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
 
Back
Top