ResponseStatusLine protocol violation

I parse/scrape a few web page every now and then and recently ran across an error that stated:
"The server committed a protocol violation. Section=ResponseStatusLine".

Exception

 Cause?:
On this page, I found an answer that suggests simply that the server header does not conform to the HTTP v1.1 specification.
[Paraphrased...]


This issue is caused when a response followed by a request to {certain} services violate the HTTP v1.1 specification.
As an example, HTTP header keys should specifically not include any spaces in their names.
Some web servers do not fully respect this which causes this error to occur.

The Microsoft .Net framework enforces the HTTP v1.1 specification and will reject any responses that do not adhere to the specification.
When a validation failure occurs then a protocol violation error is generated.

To resolve this issue any gateway/proxy level network components should been upgraded or corrected to properly adhere to the HTTP v1.1 specification.
 

 

After a few web searches, I found a couple of suggestions – one of which said the problem could be fixed by changing the HttpWebRequest ProtocolVersion to 1.0 with the command:

   1:  HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(strURI);
   2:  req.ProtocolVersion = HttpVersion.Version10;

 

 

ProtocolVersion

…but that did not work in my particular case.

 

What DID work was the next suggestion I found that suggested the use of the setting: “useUnsafeHeaderParsing” either in the app.config file or programmatically.

If added to the app.config, it would be:

   1:  <!-- after the applicationSettings -->
   2:  <system.net>
   3:      <settings>
   4:        <httpWebRequest useUnsafeHeaderParsing ="true"/>
   5:      </settings>
   6:  </system.net>

 

 

If done programmatically, it would look like this:


C++:

   1:  // UUHP_CPP.h
   2:  #pragma once
   3:  using namespace System;
   4:  using namespace System::Reflection;
   5:   
   6:  namespace UUHP_CPP
   7:  {
   8:      public ref class CUUHP_CPP
   9:      {
  10:      public:
  11:          static bool UseUnsafeHeaderParsing(String^% strError)
  12:          {
  13:              Assembly^ assembly = Assembly::GetAssembly(System::Net::Configuration::SettingsSection::typeid); //__typeof
  14:              if (nullptr==assembly)
  15:              {
  16:                  strError = "Could not access Assembly";
  17:                  return false;
  18:              }
  19:   
  20:              Type^ type = assembly->GetType("System.Net.Configuration.SettingsSectionInternal");
  21:              if (nullptr==type)
  22:              {
  23:                  strError = "Could not access internal settings";
  24:                  return false;
  25:              }
  26:   
  27:              Object^ obj = type->InvokeMember("Section",
  28:                  BindingFlags::Static | BindingFlags::GetProperty | BindingFlags::NonPublic,
  29:                  nullptr, nullptr, gcnew array<Object^,1>(0));
  30:   
  31:              if(nullptr == obj)
  32:              {
  33:                  strError = "Could not invoke Section member";
  34:                  return false;
  35:              }
  36:   
  37:              FieldInfo^ fi = type->GetField("useUnsafeHeaderParsing", BindingFlags::NonPublic | BindingFlags::Instance);
  38:              if(nullptr == fi)
  39:              {
  40:                  strError = "Could not access useUnsafeHeaderParsing field";
  41:                  return false;
  42:              }
  43:   
  44:              if (!(bool)fi->GetValue(obj))
  45:              {
  46:                  fi->SetValue(obj, true);
  47:              }
  48:   
  49:              return true;
  50:          }
  51:      };
  52:  }

C# (CSharp):

   1:  using System;
   2:  using System.Reflection;
   3:   
   4:  namespace UUHP_CS
   5:  {
   6:     public class CUUHP_CS
   7:     {
   8:        public static bool UseUnsafeHeaderParsing(ref string strError)
   9:        {
  10:           Assembly assembly = Assembly.GetAssembly(typeof(System.Net.Configuration.SettingsSection));
  11:           if (null == assembly)
  12:           {
  13:              strError = "Could not access Assembly";
  14:              return false;
  15:           }
  16:   
  17:           Type type = assembly.GetType("System.Net.Configuration.SettingsSectionInternal");
  18:           if (null == type)
  19:           {
  20:              strError = "Could not access internal settings";
  21:              return false;
  22:           }
  23:   
  24:           object obj = type.InvokeMember("Section",
  25:              BindingFlags.Static | BindingFlags.GetProperty | BindingFlags.NonPublic,
  26:              null, null, new object[] { });
  27:   
  28:           if (null == obj)
  29:           {
  30:              strError = "Could not invoke Section member";
  31:              return false;
  32:           }
  33:   
  34:           // If it's not already set, set it.
  35:           FieldInfo fi = type.GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic | BindingFlags.Instance);
  36:           if (null == fi)
  37:           {
  38:              strError = "Could not access useUnsafeHeaderParsing field";
  39:              return false;
  40:           }
  41:   
  42:           if (!Convert.ToBoolean(fi.GetValue(obj)))
  43:           {
  44:              fi.SetValue(obj, true);
  45:           }
  46:   
  47:           return true;
  48:        }
  49:     }
  50:  }

 

 


F# (FSharp):

   1:  namespace UUHP_FS
   2:  open System
   3:  open System.Reflection
   4:      module CUUHP_FS =
   5:          let UseUnsafeHeaderParsing(strError : byref<string>) : bool  =
   6:              //
   7:              let assembly : Assembly = Assembly.GetAssembly(typeof<System.Net.Configuration.SettingsSection>)    
   8:              if (null = assembly) then
   9:                  strError <- "Could not access Assembly"
  10:                  false
  11:              else
  12:              
  13:              let myType : Type = assembly.GetType("System.Net.Configuration.SettingsSectionInternal")
  14:              if (null = myType) then
  15:                  strError <- "Could not access internal settings"
  16:                  false
  17:              else
  18:              
  19:              let obj : Object = myType.InvokeMember("Section", BindingFlags.Static ||| BindingFlags.GetProperty ||| BindingFlags.NonPublic, null, null, Array.zeroCreate 0)
  20:              if (null = obj) then
  21:                  strError <- "Could not invoke Section member"
  22:                  false
  23:              else
  24:              
  25:              // If it's not already set, set it.
  26:              let fi : FieldInfo = myType.GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic ||| BindingFlags.Instance)
  27:              if(null = fi) then
  28:                  strError <- "Could not access useUnsafeHeaderParsing field"
  29:                  false
  30:              else
  31:              
  32:              if (not(Convert.ToBoolean(fi.GetValue(obj)))) then
  33:                  fi.SetValue(obj, true)
  34:                  
  35:              // Now return true
  36:              true           

VB (Visual Basic):

   1:  Option Explicit On
   2:  Option Strict On
   3:  Imports System
   4:  Imports System.Reflection
   5:   
   6:  Public Class CUUHP_VB
   7:      Public Shared Function UseUnsafeHeaderParsing(ByRef strError As String) As Boolean
   8:   
   9:          Dim assembly As [Assembly]
  10:          assembly = [assembly].GetAssembly(GetType(System.Net.Configuration.SettingsSection))
  11:   
  12:          If (assembly Is Nothing) Then
  13:              strError = "Could not access Assembly"
  14:              Return False
  15:          End If
  16:   
  17:          Dim type As Type
  18:          type = [assembly].GetType("System.Net.Configuration.SettingsSectionInternal")
  19:          If (type Is Nothing) Then
  20:              strError = "Could not access internal settings"
  21:              Return False
  22:          End If
  23:   
  24:          Dim obj As Object
  25:          obj = [type].InvokeMember("Section", _
  26:              BindingFlags.Static Or BindingFlags.GetProperty Or BindingFlags.NonPublic, _
  27:          Nothing, Nothing, New [Object]() {})
  28:   
  29:          If (obj Is Nothing) Then
  30:              strError = "Could not invoke Section member"
  31:              Return False
  32:          End If
  33:   
  34:          ' If it's not already set, set it.
  35:          Dim fi As FieldInfo
  36:          fi = [type].GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic Or BindingFlags.Instance)
  37:          If (fi Is Nothing) Then
  38:              strError = "Could not access useUnsafeHeaderParsing field"
  39:              Return False
  40:          End If
  41:   
  42:          If (Not Convert.ToBoolean(fi.GetValue(obj))) Then
  43:              fi.SetValue(obj, True)
  44:          End If
  45:   
  46:          Return True
  47:      End Function
  48:  End Class

 

 

posted @ Friday, December 3, 2010 11:10 AM
Print

Comments on this entry:

# re: ResponseStatusLine protocol violation

Left by Jim Lawless at 12/3/2010 6:13 PM
Gravatar
Heya, Tom. Do you have any idea what the issue was that caused the protocol violation?

# re: ResponseStatusLine protocol violation

Left by Tom Hines at 12/6/2010 10:46 AM
Gravatar
Jim, I just added a potential cause to the posting. It sounds like the combination of an improperly formatted header (according to HTTP v1.1) and the use of .NET to parse the page.

# re: ResponseStatusLine protocol violation

Left by Vishal Sharma at 2/26/2014 7:33 AM
Gravatar
Solution mentioned at below post helped us.

http://www.webmonkeys.org.uk/2012/09/c-the-server-committed-a-protocol-violation-sectionresponsestatusline/#comment-60.

We set "request.KeepAlive = false;" and it all worked flawlessly.

# re: ResponseStatusLine protocol violation

Left by felix at 4/28/2014 6:31 AM
Gravatar
very good.

# re: ResponseStatusLine protocol violation

Left by Daniel Peterson at 11/24/2014 1:25 PM
Gravatar
Is there anyway that I can get around this with Invoke-WebRequest? I want to run a simple 'wget' against an older web interface.

# re: ResponseStatusLine protocol violation

Left by Andre St-Onge at 4/6/2015 2:35 PM
Gravatar
Is not working for me.
I use chorme or Internet Explorer to have access to my device web page and it work it give me a download automatically except when I try to do the same way in Vb.NET is not working and give me the same message
The underlying connection was closed: The connection was closed unexpectedly.
Is I don't use your code I get this message
The server committed a protocol violation. Section=ResponseStatusLine



Function WRequest(URL As String, method As String, POSTdata As String) As String

Dim responseData As String = ""


Dim strError As String = ""
If CUUHP_VB.UseUnsafeHeaderParsing(strError) Then
System.Net.ServicePointManager.Expect100Continue = False
End If
If strError.Trim.Length Then
SetText(strError)
End If

Dim hwresponse As Net.HttpWebResponse = Nothing
Try
Dim cookieJar As New Net.CookieContainer()
Dim hwrequest As Net.HttpWebRequest = Net.WebRequest.Create(URL)
With hwrequest
.ProtocolVersion = HttpVersion.Version11
.CookieContainer = cookieJar
.Accept = "*/*"
.AllowAutoRedirect = True
.UserAgent = "chrome"
.Timeout = 100000
.Accept = "text/plain"
.Method = method
.ServicePoint.Expect100Continue = False
.ServicePoint.MaxIdleTime = 100000
.KeepAlive = False
.ContentType = "text/plain"
End With

If hwrequest.Method = "POST" Then
'hwrequest.ContentType = "application/x-www-form-urlencoded"
Dim encoding As New Text.ASCIIEncoding() 'Use UTF8Encoding for XML requests
Dim postByteArray() As Byte = encoding.GetBytes(POSTdata)
hwrequest.ContentLength = postByteArray.Length
Dim postStream As IO.Stream = hwrequest.GetRequestStream()
postStream.Write(postByteArray, 0, postByteArray.Length)
postStream.Close()
End If
hwrequest.AllowWriteStreamBuffering = True

hwresponse = hwrequest.GetResponse()
If hwresponse.StatusCode = Net.HttpStatusCode.OK Then
Dim responseStream As IO.StreamReader = _
New IO.StreamReader(hwresponse.GetResponseStream())
responseData = responseStream.ReadToEnd()
End If
hwresponse.Close()
Catch e As Exception
responseData = "An error occurred: " & e.Message
End Try
Return responseData
End Function

# re: ResponseStatusLine protocol violation

Left by Raul Marmolejo at 10/13/2015 12:01 PM
Gravatar
Works great!
Can I use that class for a project under GNU GPL v2.0, mentioning you and your blog?

Thanks

# re: ResponseStatusLine protocol violation

Left by Krug Kane at 10/15/2015 5:35 AM
Gravatar
Hi,
it doesn't work for me. I wrote a very simple PHP web service using NuSoap library. I need to wait about 10 seconds since the last call to the web service or I get the protocol violation error. How can I fix it?
Regards,
Krug.

Your comment:



(not displayed)

 
 
 
 

Live Comment Preview: