|
I have the domain www.mydomain.com and I set apache mod-rewrite so as to have www.mydomain.com/myappl. Where should I place the file robots.txt? thanks!
Started by Angelo on
, 3 posts
by 3 people.
Answer Snippets (Read the full thread at stackoverflow):
So if you don't know guess where.
How ;-) But you can use the robots checker to find if your file is accessible.
The robots.txt must be accessible to clients as http://www.mydomain.com/robots.txt .
|
|
I'm reviewing a couple of my web sites to make sure my SEO bases are covered. There are no private pages on the site in question, and we want all pages to be indexed. Does including a robots.txt file anyway, even if it isn't needed, make my site look ...
Started by Stuart B on
, 6 posts
by 6 people.
Answer Snippets (Read the full thread at stackoverflow):
Google has....
If it's not there, the search engineno, it won't.
Search engines will look for a robots.txt file.
Their instructions / guidelines are actually pretty thorough.
As per Google , it's a good idea.
robots.txt file.
|
|
I've got a Plone site that I administer and I'd like to add some pages to the Disallow of a robots.txt.
It appears that Plone automatically generates a robots.txt file. I can't find any way to modify that. I've also tried adding a 'robots.txt' file to...
Started by Jimmy Z on
, 3 posts
by 3 people.
Answer Snippets (Read the full thread at stackoverflow):
Using the ZMI....
Create a robots.txt on your desktop Go in the xmi of the plone site -> add a new file -> call#nabble-td329779
As has been noted already, Plone 3.x already includes a robots.txt file so the preferred one.
|
Ask your Facebook Friends
|
Robots.Txt File Hello Friends,
I would like to know that what is Robots.txt file and how it should be created? Does it necessary for search engine ranking? Please reply. Please explain with an example if possible.
I want make this for http://www.preventionofdisaster...
Started by pranav9189 on
, 18 posts
by 18 people.
Answer Snippets (Read the full thread at webdevforums):
User-Agent regarding....
It from TheWebPole.com Robots.txt "Robots.txt" is a notepad file that contains those webpages you do not want of your site.you can create robots.txt file using notepad with just two lines below code..
|
|
Robots.txt file I would like to know that which things of website, an expert SEO must put into robots.txt file for disallow? Which pages is better to not show to Search Engines? Android Application Development | iPhone App Development | Android App Developer...
Started by jesicawillss on
, 12 posts
by 11 people.
Answer Snippets (Read the full thread at webdevforums):
Disallow admin control page in robots.txt file, but there should be more pages those having since-less to robots.txt file where declared that which folder or file are allow to search engine indexing and which of website....
|
|
I have a bunch of files at www.example.com/A/B/C/NAME (A,B,C change around, NAME is static) and I basically want to add a command in robots.txt so crawlers don't follow any such links that have NAME at the end.
What's the best command to use in robots...
Started by Mike F on
, 7 posts
by 6 people.
Answer Snippets (Read the full thread at serverfault):
To my knowledge there is no pattern matching routine supported by the robots.txt file parsers in mind that listing those files in the robots.txt file will give out a list of those links to anyone who so it executes....
|
|
Looks like we are going to have to start load balancing our webservers here soon.
We have a feature request to edit robots.txt dynamically which is not a problem for one host -- however once we get our load balancer up and going -- it sounds like I will...
Answer Snippets (Read the full thread at stackoverflow):
It makes a lot of sense since robots.txt is going to be requiredI might have the app edit the contents of robots.txt and have the user input saved to a database robots.txt differently.
For that file go to a single server.
|
|
What is a robots.txt file? i have heard this mentioned a few times, does it help search engines find your site and the subpages? if so how do i make one and where do i put it.
I use front page, and hostmonster.com.
my page is not a frames page but it ...
Started by Leadsled on
, 16 posts
by 16 people.
Answer Snippets (Read the full thread at webdevforums):
Can get it from XnYnZ.com Robots.txt is a text file you put on your site to tell search robots Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would Hosting Robots.txt....
|
|
Use of Robots.txt Why we use Robots.txt File? web designing ludhiana
Bulk SMS Ludhiana
Started by dimpu on
, 15 posts
by 15 people.
Answer Snippets (Read the full thread at webcosmoforums):
SEO Services SEO Company A "robots.txt" file is a text file placed on your server which contains a list of robots and "disallows to be created you....
To inform search engines about your will is to use a robots.txt file.
|
|
** I changed the subject header in order to point people towards the info re .htaccess and robots.txt files that is contained later in this thread. **
Another edit - January 19, 2004 - removed my rant - it isn't needed BUT all this information is good...
Answer Snippets (Read the full thread at menalto):
Anyway.....
Those robots.txt file, in case anyone isn't sure where to start or HOW LARGE they can be (yeah, this one is kind long).
My .htaccess file and my robots.txt keep a lot of the riff raff away.
Hi
No - thanks.
For you.
|
