WebRequest is the abstract base class for the .NET Framework's request/response model for accessing data from the Internet.
To get content of a website, in .NET 1.0. we used to use WebRequest, which is good and also works asynchronously.
public static string GetContent(string url)
{
System.Net.WebRequest request = System.Net.WebRequest.Create(url);
using (System.Net.WebResponse response = request.GetResponse())
{
using (System.IO.StreamReader reader =new System.IO.StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
}
But in .NET 2.0, we can also use the WebClient class. It can also work asynchronous and works the same as the other one.
public static string GetContent(string url)
{
using (System.Net.WebClient client =new System.Net.WebClient())
{
return client.DownloadString(url);
}
}
We can use any of the above method for web scrapeing in .NET. But the second approach is probably more cleaner.