Ask questions via twitter! Message any question to @answers on twitter. We'll publish the question and send you a reply each time there's a new answer.
Next Question

Answered Question

dglp dglp
 
December 15, 2008 09:36 PM

What search engine/method will yield files in website subdirectories?

Suppose I want to find all the files in subdirectories of www.site.net, e.g. files at www.site.net/sub1/sub2/*.*. The major search engines don't yield results from anything but the root. Is there a reliable method of searching subdirectories on a given site?
Interesting Question?  Yes (0)   No (0)   
RSS
 
 

Best Answer  Chosen by Asker

tko tko
 
 December 16, 2008 01:08 AM
All of the search engines parse subdirectories. Provided those subdirectories are linked from somewhere, and don't have code in them, or on the site telling search engines *not* to parse them.

Search engines are big money these days. They'll search as exhaustively as their programmers can devise. (Heck, look at the amount of PDF's that are searchable ..image and audio recognition are going to start playing a big part soon too. Soon there won't be much you *can't* find.)


Helpful Answer?  (0)   (0)    Tip tko for this answer
Permalink | Report
   Reply  
 
 
dglp dglp
 
December 19, 2008 09:21 PM
I would like to give this a tie for 'best answer' since it answers part of my question very clearly and emphatically.

Report
 
 

Other Answers (3)

Sort By
jeffhoard jeffhoard
 
December 15, 2008 09:40 PM
I think you might have luck if you go to Google.com and type in your search query followed by site:websitetitlehere.com

So if you wanted to find all the pages on Mahalo related to Sarah Palin you would get these results...

http://www.google.ca/search?hl=en&q=sarah+palin+site%3Amahalo.com&btnG=Google+Search&meta=

Helpful Answer?  (0)   (0)    Tip jeffhoard for this answer
Permalink | Report
   Reply  
 
 
dglp dglp
 
December 15, 2008 09:52 PM
Very fast result Jeff. I had tried this various times in the past, and didn't get results. But I've just tried it in 3 places and it's worked on two of them. The 3rd one seems to have locked subdirectories - and maybe that's why the search fails.

Report
 
 
dglp dglp
 
December 19, 2008 09:43 PM
This is the best answer in that it provides a method and a direct answer to the question.

Report
 
 
loopy1 loopy1
 
December 15, 2008 09:58 PM
ALL of them will, as long as the website doesn't restrict the search engine from looking in subdirectories with a restrictive robots.txt file. For example, I did a search on google for chicagotribune.com, and the third result was archives.chicagotribune.com/2008/nov/11/health/chi-081111bishops. While it's not technically a subdirectory (for the reason that most modern websites don't run off of flat html files that are stored in a directory structure) it may be the type of result you are looking for.
Source(s):
Experience.


Helpful Answer?  (0)   (0)    Tip loopy1 for this answer
Permalink | Report
   Reply  
 
 
ilaksh ilaksh
 
December 15, 2008 10:01 PM
You could use a sitemap generator like http://www.xml-sitemaps.com . xml-sitemaps.com has options to download the sitemaps in html, xml or text lists of urls. Or try http://www.sitemapdoc.com/

Helpful Answer?  (0)   (0)    Tip ilaksh for this answer
Permalink | Report
   Reply  
 
 
dglp dglp
 
December 19, 2008 09:17 PM
This is my favourite answer among the first four, because it gave me an alternative way of looking. Very useful!

Report
 
 

Answer this Question

How tips and payments work

This question has already been resolved. You may add an answer to it but you will not be eligible to win best answer or any associated tips.

Related Questions

View All Search Engines Questions

Ask a Question


140 characters left
Top of Page
Buy Mahalo Dollars with Credit Card or PayPal

Top Members

This Week All Time
  • buddawiggi
    buddawiggi
    2nd Degree Black Belt
    27933 Points
    M$806.66 Earned
  • opher
    opher
    Purple Belt
    4757 Points
    M$203.72 Earned
  • annelisle
    annelisle
    Purple Belt
    3308 Points
    M$99.72 Earned
   See All
 

Most Popular Tags

mahalo(1638)
twitter(957)
iphone(467)
music(464)
google(361)
food(326)
internet(310)
online(298)
beer(280)
money(267)
movies(265)
facebook(256)
apple(254)
aotd(235)
answers(231)
health(220)
video(209)
free(206)
dog(205)
windows(198)
   See All
 

Categories

Welcome New Members


 
 
Mahalo Dollars are the currency of Mahalo Answers.

Each Mahalo Dollar costs $1.

Once you earn more than 40 Mahalo Dollars, you can request to be paid via PayPal. Each Mahalo Dollar is currently worth $0.75 when paid out via PayPal. Learn More

 
 

Please log in to use this function.

Answers Home | Ask a Question | Earn Points | How Tips Work | Widget