One of the things I always forgot to add to my web applications is the Robots.txt file that Search Engines use to see what they should index. This file and site maps help make your site easier to navigate by the bots and let them know what is legal and what you would rather not have the published in their engines. I typically add any administrative pages or account pages even though they are protected by security, no need for the login page to be index if they sniff the link.
So how do you add Robots.txt to your MVC 3 application? Glad you asked, here is a little code to get you started.
Sample Code
1. Select the controller you would like to use for the robots.txt output. I chose the HomeController in my application as I use it for most “top level” generic links like about us, contact us, index, etc.
2. Create a method called Robots to handle the request.
#region -- Robots() Method --
public ActionResult Robots()
{
Response.ContentType = "text/plain";
return View();
}
#endregion
3. Add the Robots.cshtml view to your Controller’s View directory. Here is the code I have in my view, yours will vary.
@{
Layout = null;
}
# robots.txt for @this.Request.Url.Host
User-agent: *
Disallow: /Administration/
Disallow: /Account/
4. Load up the class you are using to control your routes, if you are in an Area, this could your AreaRegistration class. If you are at the top like I am and using the standard MVC template, this is probably the Global.asax.cs file. Add your route to this file, mine looks like this.
routes.MapRoute("Robots.txt",
"robots.txt",
new { controller = "Home", action = "Robots" });
5. Compile and test.
Conclusion
If you have an internet facing site, the chances are you will have a bot find you are request this page. You might as well give them the benefit of the doubt and let them know where you want them to go. Also you will save yourself some error log when this page is requested and no controller is found.
Just like anything in ASP.NET, there are many ways to solve this riddle, if you use a different approach, please feel free to share it in the comments.
Awesome news! The TechEd 2012 Closing Party is at Universal’s Island of Adventure theme park this year. The party is on Thursday, which give you a reason not to leave the conference early and actually catch some of the repeat sessions that you missed from crowds.
I am really excited to check out the Harry Potter area of the park, I have heard great things. John and I will be making a presence again at TechEd, stay tuned for more details.
Links:
I have to admit, I am pretty excited about this. I am not sure about the dates, but the Oak Park Mall in Overland Park, KS will be getting a Microsoft Store very soon based on this article. I am guessing this is due to the amount of Best Buy stores that are closing.
I have never been to a store, but I am excited, what has everyone else thought about the experience?
It has been a week since we moved our Geekswithblogs.net System to a new set of load balanced servers and everything has been going great. I am so amazed at the performance of the new hardware. On average, we only use less than 5% of the CPU at any given moments or the database and web servers. I have seen a performance boost in page load as well, but I will have to confirm that with the statistics as they roll in.
This is all in preparation for a new community we are launching with some friends that we will be announcing shortly. We will be launching a nice little contest for our bloggers as well.
Well I like to look at pretty pictures on deployment days to keep my mind on the good side of software. Well I guess it is not that bad, but today we moved Geekswithblogs.net to a new server pool to update hardware, get some extra machines in the mix, and move to SQL Server 2008 R2. If you see any issues, please alert us directly via comments or support@geekswithblogs.com. Thanks and hopefully we will see more benefits that issues.
Red Gate launched the Ants Performance Profiler 7.0 product today. This application is simply amazing! SharePoint 2010 support! Decompilation without PDB using Reflector’s engine! Azure and EC2 Support! Seriously, if you need a profiler for your .NET applications this one takes the cake.
They have some great videos online to show you how the product works and the team is very interactive if you have questions or concerns. There is also a 14 day trial so you can give it a go. The license fee is only $395, which is amazing if you think about the time it could save you on finding out why your app is not performing well.
Here some details I received from the product team:
Version 7.0 includes integrated decompilation: when profiling methods and assemblies with no source code file, you can generate source code right from the profiler interface. You can then browse and navigate this automatically generated source as if it was your own. If you have an assembly’s PDB file but no source, integrated decompilation even lets you view line-level timings for each method, pinpointing the exact cause of performance bottlenecks. Integrated decompilation is powered by .NET Reflector, but you don’t need Reflector installed to use the functionality.
Also new in ANTS Performance Profiler 7.0:
· Full support for SharePoint 2010 – No need to manually configure profiling for the latest version of SharePoint
· Full support for IIS Express
· Azure and Amazon EC2 support, enabling you to profile in the cloud
Check out the site to learn more and feel free to let me know what you think of the product!
James Hare did a review of 6.3 last year if you are unfamiliar with the product.
One of the great things about 2012 is that I am not in our office nearly 100% of the time during business hours. This is the first time I have been able to work from our office this much in about 4 years. In fact, we have moved three times during that four years and I have only averaged about 10-25% of my time in the office.
This means I can actually setup my desk for development instead of business related tasks only. That means THREE MONITORS! My first reaction after a week with this setup: I never want to go back to two monitors.
Visual Studio is a beast when it comes to multi-monitor support. It remembers where you placed your floating windows and keeps track of all the other goodies. I do wish I could detach a document and then make it dockable by other monitors, but at least with Windows+Left and Windows+Right and I quickly tile the documents on the external monitors.
So far with an MVC application I have been working on I was able to have SQL Server, Visual Studio, detached Solution/Property/Team Explorer and my browsers all open at the same time. I can already find a use for going with 6 monitors now, but that will have to wait…
I am definitely excited to see this new setup working for some of our SharePoint consulting and Team Foundation Server help we do. Those projects always have a need for multiple applications and flipping between quickly to get the job done.
Kevin Tews, a very awesome guy who was the organizer of my trip to Haiti two years ago put together and amazing video from some of the photos I took while in Haiti. Even today, I have still not got back to spunk I used to have for photography before this trip. When you see this through the lens, it is hard to want to take a picture of a park or cityscape. Seeing the devastation and the beauty of Haiti definitely changes a person.
Link: http://youtu.be/McZDLjUKJ-c
Technorati Tags:
Haiti,
2 Years Later
I have always been a fan of Google Reader. I have been using it since it first came out and have never tried another feed reader. Yesterday they decided to update the UI to fit their new G+ look and in my opinion, made the application almost impossible to use.
I have three main machines I use on a daily basis. A Dell XPS desktop with dual 24 inch monitors, a Toshiba Protégé with a 13 inch display, and a 11 inch MacBook Air. When I am at my desk all day, the Dell is awesome for getting development done, but I don’t like to clutter my noise-generating applications on my development machine so I leave Outlook, Thunderbird, Messenger, Zune, and my social browsing to the Toshiba with 1366x768 resolution. This resolution is standard for most < 15 inch laptops.
Now when you open up Google Reader on the laptop you get this huge portion at the top of the page that is wasted whitespace. Big dumb buttons with small text and lots of padding and a toolbar with lots of padding. On a big monitor with 1920x1080 resolution, its ok, but on the small monitor, you are left with half of the page to read your post and the other half for this dumb whitespace.
Any who, if anyone has an alternative Feed Reader they are using, please let me know.
Ok, I think this is the coolest contest I have seen. Red-Gate is sending a DBA to Space! If you have mad SQL skills, you should definitely look into this contest.
More Information:
Red Gate, a U.K.-based software company, has launched a campaign that will reward a database administrator (DBA) with the ultimate getaway: a seat on a commercial flight into space, courtesy of Space Adventures, Ltd
But first, DBAs must unravel a Gordian knot of a plot involving morphing Martians, pets in peril, alien body parts and abduction of one of their own – all unfolding in a five-week series of snazzy (and frankly surreal)
60s-style space-noir B-movies.
Fifteen finalists will be selected from those who provide the best answers to quizzes within the videos and the ripest tweets from dataspace. The winning DBA will be elected by popular vote to join the first citizens of Earth to sail beyond our atmosphere.
Visit https://www.dbainspace.com/ to learn more about the contest!