Omgili - forum search, search forums  
  

Discussions about robots.txt file

Displaying 1 - 10 out of 981 discussions.  
Time Frame: (Any time)   Minimum number of replies: (2)   Minimum number of discussing users: (0)
  |  

Page: 1   2   3   4   5   6   7   8   9   10  
Keep this page open to be updated with the newest discussions automatically.
I have the domain www.mydomain.com and I set apache mod-rewrite so as to have www.mydomain.com/myappl. Where should I place the file robots.txt? thanks!
Started by on , 3 posts by 3 people.  
So if you don't know guess where.
How ;-) But you can use the robots checker to find if your file is accessible.
The robots.txt must be accessible to clients as http://www.mydomain.com/robots.txt .
I'm reviewing a couple of my web sites to make sure my SEO bases are covered. There are no private pages on the site in question, and we want all pages to be indexed. Does including a robots.txt file anyway, even if it isn't needed, make my site look ...
Started by on , 6 posts by 6 people.  
Google has....
If it's not there, the search engineno, it won't.
Search engines will look for a robots.txt file.
Their instructions / guidelines are actually pretty thorough.
As per Google , it's a good idea.
robots.txt file.
I've got a Plone site that I administer and I'd like to add some pages to the Disallow of a robots.txt. It appears that Plone automatically generates a robots.txt file. I can't find any way to modify that. I've also tried adding a 'robots.txt' file to...
Started by on , 3 posts by 3 people.  
Using the ZMI....
Create a robots.txt on your desktop Go in the xmi of the plone site -> add a new file -> call#nabble-td329779 As has been noted already, Plone 3.x already includes a robots.txt file so the preferred one.
Ask your Facebook Friends
Robots.Txt File Hello Friends, I would like to know that what is Robots.txt file and how it should be created? Does it necessary for search engine ranking? Please reply. Please explain with an example if possible. I want make this for http://www.preventionofdisaster...
Started by on , 18 posts by 18 people.  
User-Agent regarding....
It from TheWebPole.com Robots.txt "Robots.txt" is a notepad file that contains those webpages you do not want of your site.you can create robots.txt file using notepad with just two lines below code..
Robots.txt file I would like to know that which things of website, an expert SEO must put into robots.txt file for disallow? Which pages is better to not show to Search Engines? Android Application Development | iPhone App Development | Android App Developer...
Started by on , 12 posts by 11 people.  
Disallow admin control page in robots.txt file, but there should be more pages those having since-less to robots.txt file where declared that which folder or file are allow to search engine indexing and which of website....
I have a bunch of files at www.example.com/A/B/C/NAME (A,B,C change around, NAME is static) and I basically want to add a command in robots.txt so crawlers don't follow any such links that have NAME at the end. What's the best command to use in robots...
Started by on , 7 posts by 6 people.  
To my knowledge there is no pattern matching routine supported by the robots.txt file parsers in mind that listing those files in the robots.txt file will give out a list of those links to anyone who so it executes....
Looks like we are going to have to start load balancing our webservers here soon. We have a feature request to edit robots.txt dynamically which is not a problem for one host -- however once we get our load balancer up and going -- it sounds like I will...
Started by on , 4 posts by 4 people.  
It makes a lot of sense since robots.txt is going to be requiredI might have the app edit the contents of robots.txt and have the user input saved to a database robots.txt differently.
For that file go to a single server.
What is a robots.txt file? i have heard this mentioned a few times, does it help search engines find your site and the subpages? if so how do i make one and where do i put it. I use front page, and hostmonster.com. my page is not a frames page but it ...
Started by on , 16 posts by 16 people.  
Can get it from XnYnZ.com Robots.txt is a text file you put on your site to tell search robots Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would Hosting Robots.txt....
Use of Robots.txt Why we use Robots.txt File? web designing ludhiana Bulk SMS Ludhiana
Started by on , 15 posts by 15 people.  
SEO Services SEO Company A "robots.txt" file is a text file placed on your server which contains a list of robots and "disallows to be created you....
To inform search engines about your will is to use a robots.txt file.
** I changed the subject header in order to point people towards the info re .htaccess and robots.txt files that is contained later in this thread. ** Another edit - January 19, 2004 - removed my rant - it isn't needed BUT all this information is good...
Started by on , 41 posts  
Answer Snippets (Read the full thread at menalto):
Anyway.....
Those robots.txt file, in case anyone isn't sure where to start or HOW LARGE they can be (yeah, this one is kind long).
My .htaccess file and my robots.txt keep a lot of the riff raff away.
Hi No - thanks.
For you.
Page: 1   2   3   4   5   6   7   8   9   10  

Related Message Boards & Forums

  • Stack Overflow
  • Search Engine Optimization and Marketing
  • Customizing Gallery 1 | Gallery
  • Server Fault
  • Search Engine Optimization
More Information


Forum Search About Omgili Help Plugins Forum/Board Owners Privacy

i
In Title
In Topic
In Reply
Exclude
Boost