We've moved.
Visit our new home for news and updates from AdSense
Inside AdSense
Google's official blog for news, tips and information on AdSense.
I, Robots.txt
Tuesday, February 07, 2006
Often when troubleshooting targeting inquiries, we find that publishers have unknowingly blocked the
Googlebots
from crawling sections of their site, resulting in PSAs or mistargeted ads. With the newest release of
Google Sitemaps
, you can get a report that shows you Googlebot's view of your
robots.txt file
.
In addition to general status updates such as the last time Googlebot downloaded your robots.txt file, this new report allows you to test specific URLs to see if your robots.txt file is blocking them from the MediaPartners Googlebot, the Google content crawler. Plus, you can test changes to your robots.txt file against other Google crawlers (e.g., search or image) or the Robots Standard. This way you can experiment with changes to see how they would impact the crawl of your site, as well as make sure there are no errors in the file, before making changes to the file on your site.
Learn more
about this recent release on the Google Sitemaps Blog.
Still not sure if Sitemaps can really help you? Check out the newest Sitemaps
success story
showcasing fellow AdSense publisher ApartmentRatings.com.
Posted by Shaluinn Fullove - Sitemaps Product Marketing
Labels
#GlobalSpotlight
2
#SuccessStack
9
Active View
9
Ad formats
22
Ad layout
6
Ad Placement
2
AdSense
5
AdSense Anniversary
15
AdSense features
231
adsense guide
14
AdSense In Your City
22
AdSense Interface
61
AdSense Team
31
AdSense101
13
AMP
3
AskAdSense
4
Audience Engagement
4
Beginner's
1
Best Practice
22
Beta
1
Case studies
99
Certified Partners
4
Content strategy
23
DoubleClick
17
download
3
Earnings/Reports
70
eBook
3
Education
4
engagement
1
Facts and Fiction
11
FAQs
1
Feedback
10
Google Analytics
7
Google's Certified Publishing Partner
3
googlenew
13
Guest Blogger
4
Help
2
How to avoid policy violations
1
India
1
infographic
3
International
118
Learn with Google for Publishers
17
Matched content
6
micro-moments
5
Mobile
67
Mobile optimization
3
Multi-Screen
32
Native
1
native ads
4
Newbie
32
Optimal ad placement
1
Optimization
165
Other
245
Other Google products
106
Page-Level
1
PageSpeed
10
Pakistan
1
Passion into profit
8
Payments
62
Policy
2
Pop-Unders
1
Pop-Ups
1
Product update
11
Program Policies
79
Programmatic
1
Social
19
Support
10
Tips
37
Urdu
1
User Generated Content
1
Video
41
Viewability
13
Webinars
15
YouTube
3
Archive
2019
Oct
Sep
Aug
Jul
May
Feb
Jan
2018
Dec
Nov
Oct
Aug
Jun
May
Apr
Mar
Feb
Jan
2017
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2007
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2006
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2005
Dec
Nov
Oct
Sep
Aug
Feed
Subscribe to receive posts via email:
Visit this group
Follow @adsense
Useful links
Get started with AdSense
Sign in to your account
AdSense Help Center
AdSense Help Forum
Related Google products
AdMob: Monetize your app
DoubleClick for Publishers Small Business
Google AdWords
Google Analytics
Give us feedback in our
Product Forums
.
