Last week I pushed a new application up to production, and started hearing complaints of a memory leak. After running some diagnostics I learned two exciting things:
http://bugs.jqueryui.com/ticket/7666
The current version of jQuery UI (1.18.16) has a memory leak with the DatePicker control - including the DatePicker control as part of the jQuery UI is all it takes to cause memory to be allocated and never returned until the browser is closed. Every refresh of the page, or every time a new page is loaded, more memory is allocated to the browser process.
Although my application did not utilize the DatePicker control, it came as part of the default jqueryui package and I did not go out of my way to exclude the DatePicker from the jqueryui package.
This reminded me of an important aspect of software design: The more features a system has, the more features a system has that can go wrong. Although today we may believe our system to be safe and secure, new vulnerabilities will undoubtably be found tomorrow proving that we weren't so safe after all. Exposing additional functionality that is not intended to be used is potentially dangerous and in my case caused an unecessary memory leak.
At the same time I discovered another memory leak in my implementation of http://datatables.net/ that applies to IE 7.
I had a table like such:
$(document).ready(function() {
$('#example').dataTable( {
"aoColumns": [
{ "fnRender": function ( oObj ) {
return "<div onMouseOver=\"alert('abc');\">" +
oObj.aData[0] + "</div>"; } }
]
} );
} );
(In other words, I added some basic javascript functionality at render time to the outputed html of my datatables.)
I noticed that when I would page forwards and backwards through my datatable, memory was being leaked within Internet Explorer - In the html of the table, we have a javascript onMouseOver event which was being registered against the DOM, but since the datatables library did not do the registering, it did not know to unregister the event before loading the next page of data.
Instead of registering the events to the cells themselves, I switched to event delegation (http://www.quirksmode.org/blog/archives/2008/04/delegating_the.html) which is a much better strategy than registering an event listener for every row of a table that could potentially be several hundred rows long.
Two memory leaks down. How many more could there possibly be?...
With SSRS we can quickly generate reports that can be exported to multiple formats. What happens when you want to extend your report with custom code?
Friday November 11th 2011 (this Veteran's Day) I'll be talking about just that in Eden Prairie MN during:
http://sqlsaturday.com/99/eventhome.aspx
Perhaps you need a custom authentication layer, custom access to data requiring .Net code, or you want to extend some of the controls that come out of the box with SSRS. In this session we will see an example of extending SSRS to use a .Net library for a data provider allowing us to use a custom .Net business layer for our report. We will also show an example extension to the SSRS graph control allowing us to make design graphs for SSRS. Custom code is a great alternative for those considering using toolkits like Nevron or Dundas toolkits to extend their SSRS graphing capabilities
SSRS is a wonderful tool for quickly retrieving data from many different data sources and presenting the data to the user at a run-time decided format. One area where SSRS often falls short is when the underlying data needs to come from several different sources. Perhaps we want to retrieve data from the General Ledger which is in Oracle, and join that against a list of departments and employees which are stored in SQL Server for us to display in one table. When this happens, we are unable to join them into one dataset without the use of a linked server.
I've put together a sample SSRS DPE project and uploaded it to Code Plex as a starting point for any interested developers to work from. Depending on the specific needs of the project I'm certain additional development will be required, but my goal is to create a useful starting point for anybody that needs to use SSRS to report data coming from two or more different databases.
http://www.codeproject.com/KB/reporting-services/SSRSMultiDataSourceDPE.aspx
Javascript is a very powerful tool to enhance the user experience. However, poorly optimized javascript can have a very negative effect due to increased page load time. This becomes particularly apparent as the number of elements on a page increases thanks to controls bound to increasingly large data sets.
For those that are new to jQuery, assuming we have two divs (div_drag and div_drop), we can add drag and drop functionality by registering listeners during our form load:
$("#div_drag").draggable();
$("#div_drop").droppable({
accept: "#draggable"
});
Now instead let's assume we have two lists - active & inactive. We want to be able to drag items from left to right and back again. With a maximum height and overflow, we will have scrollbars on our list for when the length of the items extends beyond 400px.
<table>
<tr>
<td>
<div class="DrgTgtList" >
<div id="bktInactivate" class="DrgTgtHeader">
<h3>Inactive Items</h3>
(Drop here to remove an item)
</div>
<ul id="InactiveItems" class="InactiveList"
style="list-style-type:none; height: 400px; overflow:auto;"
onscroll="SetDraggable(this)">
<li class="DrgInactive" id="1">ABC</li>
<li class="DrgInactive" id="2">XYZ</li>
<li class="DrgInactive" id="3">123</li>
<li class="DrgInactive" id="4">456</li>
</ul>
</div>
</td>
<td>
<div style="DrgTgtList" >
<div id="bktActivate" class="DrgTgtHeader" >
<h3>Active Items</h3>
(Drop here to add a new item)
</div>
<ul id="ActiveItems" class="ActiveList"
style="list-style-type:none; height: 400px; overflow:auto"
onscroll="SetDraggable(this)">
<li class="DrgActive" id="7">A</li>
<li class="DrgActive" id="8">B</li>
<li class="DrgActive" id="9">C</li>
<li class="DrgActive" id="10">D</li>
</ul>
</div>
</td>
</tr>
</table>
If we wanted to make the divs draggable, all we would need is to register the divs like below:
<script language="javascript" type="text/javascript">
$(document).ready(function(){
$("#bktInactivate").droppable({
accept: ".DrgActive",
hoverClass: "drop-state-hover",
drop: function(event, ui) {
alert('deactivate!')
});
$("#bktActivate").droppable({
accept: ".DrgInactive",
hoverClass: "drop-state-hover",
drop: function(event, ui) {
alert('activate!');
}
});
$("#bktActivate li").draggable({
helper: "clone"
});
$("#bktAInactivate li").draggable({
helper: "clone"
});
}
</script>
This will work perfectly fine for a relatively small number of items in our list. However, in the above code, every div that is to be draggable or droppable is registered at the same time (the document read event). Once the number of draggable or droppable items in the list gets too high (upwards of 100) the performance of attaching the event listeners becomes very noticable in IE based browsers (several seconds to several minutes). To combat this slow, unwieldy page load, we decided to defer the event registration as long as possible.
In the below sample, we only want to register the items that are currently visible. As we scroll up and down on the containing lists, we will then register the draggable events to the newly visible items. jQuery will maintain which elements have already been registered more efficiently than we will be able to easily do so, so the responsibility of double registration is left to jQuery.
When the page loads, we will register our two droppable divs (the headers of each list) and initialize the currently visible draggable divs as draggable. Only when the scroll bar moves other divs into visibility will they be registered. We can now have a virtually unlimited number of draggable divs in our panel and feel no performance penalty on form load.
<script language="javascript" type="text/javascript">
$(document).ready(function(){
$("#bktInactivate").droppable({
accept: ".DrgActive",
hoverClass: "drop-state-hover",
drop: function(event, ui) {
alert('deactivate!')
});
$("#bktActivate").droppable({
accept: ".DrgInactive",
hoverClass: "drop-state-hover",
drop: function(event, ui) {
alert('activate!');
}
});
SetDraggable(document.getElementById('ActiveItems'));
SetDraggable(document.getElementById('InactiveItems'));
}
function SetDraggable(draggableUL) {
var scrollTop = draggableUL.scrollTop;
// Activate li items currently visible
for (var index = 0; index < draggableUL.childNodes.length; index++) {
var child = draggableUL.childNodes[index];
if (child.offsetTop < scrollTop) // skip down until we are at current level
continue;
if (child.offsetTop > scrollTop + 400) // stop after we pass visible level
break;
$("#" + child.id).draggable({
helper: "clone"
});
}
}
</script>
I am working on a project where a series of ranges are to be administered. The ranges are associated with a certain value. Say that a sales associate is at 20% commission when his sales are from 0 to 3 million, and after 3 million, the commission goes up to 25%. This data is stored in tabular format in the database, but administering the data through a tabular grid view was getting very difficult to maintain and administer. It was difficult to prevent the user from creating gaps or overlap in the ranges.
We wanted a graphical way to represent range based data. We first tried to use a "slider" (http://jqueryui.com/demos/slider/) control, but since the ranges are all affected by each other this caused us to layer sliders on top of sliders and it became a visual and administrative mess. We then went over to the white board to think of how else to represent ranges of values.
We first worked out a number line, with icons indicating the various possible actions. We tried to decide on a layout that would be as easy to reproduce in html as possible. The source code and a brief description for our Range Administration and Maintenance Control used to manage ranges of numbers is available at the following URL:
http://www.codeproject.com/Tips/232423/Range-Administration-and-Maintenance-Control
Hopefully somebody finds this a useful way to represent data, or maybe other people have put together other controls to maintain ranges of data.
At a client I work at, we have a few dozen WCF services used throughout the environment. It's become difficult to consolidate the various services so that people know when knew ones become available and to administer their connections as environments change.
To address this, we created one web project that has connection information for every service. Each service has a different factory method depending on Binding and Endpoint credentials. The bindings and endpoints are generated programmatically, rather than through the convoluted XML nodes necessary in the App.Config.
We have a few base classes BaseBinding, and BaseEndpoint:
class BaseBinding : System.ServiceModel.BasicHttpBinding
{
public BaseBinding()
{
}
}
class BaseEndpoint : System.ServiceModel.EndpointAddress
{
public BaseEndpoint(string serviceName) :
base(ConstructUri(serviceName))
{
}
private static string ConstructUri(string serviceName)
{
// Could read from config, or local resource file to determine appropriate
// addresses based on environment (prod, dev, UAT)
if (serviceName == "WEATHER")
return "http://wsf.cdyne.com/WeatherWS/Weather.asmx";
else
return "http://www.w3schools.com/webservices/tempconvert.asmx";
}
}
And also in the ServiceLibrary we have our sample classes:
public class TempuratureUtil : TempuratureService.TempConvertSoapClient
{
public TempuratureUtil() :
base(new BaseBinding(), new BaseEndpoint("Tempurature"))
{
}
}
public class WeatherUtil : WeatherService.WeatherSoapClient
{
public WeatherUtil() :
base(new BaseBinding(), new BaseEndpoint("WEATHER"))
{
}
}
Now in the application layer, I simply add a reference to the class library:
static void Main(string[] args)
{
try
{
TempuratureUtil temp = new TempuratureUtil();
string result = temp.CelsiusToFahrenheit("23");
WeatherUtil weather = new WeatherUtil();
var forecast = weather.GetCityForecastByZIP("55337");
}
catch (Exception ex)
{
string output = ex.Message;
}
}
The application is not concerned with the specifics of the WCF service, it is all determined by the class library which is being referenced.
I have a .Net 4.0 site that uses Entity Framework and Linq to SQL.
After reworking a few tables and a few page operations, a new error appeared: "Row not found or changed".
Every time I commit changes to a particular table, this error would be raised.
From a quick review of msdn and the internet, the primary reason for this exception appears to be data concurrency issues - (The source data in the database is out of sync with the data you are attempting to update.)
That did not apply to my development environment, so I went to the web for more answers:
http://social.msdn.microsoft.com/forums/en-US/linqprojectgeneral/thread/c672c8ee-bf2a-41b4-bb8b-aa76cc5d9b95/
Two posts stood out to me on this thread -
- This might be caused by concurrency checks on data that is round-tripped to the client in hidden fields and then reconstituted on the server to be used as 'original' values during the update. Some data types may loose precision when converted to text and back. You can avoid this problem by turning on concurrency checks for this field in the mapping (a propety of the field/column in the designer.)
- I have a feeling it might have something to do with a null<>string.empty conversion within the LinqDataSource control/viewstate. In my case, the original row had a null value in a nullable column. While handling the OnUpdating event to debug, it looked like it may have been stored as an empty string in the original object.
I then compared the object directly with the data in the database - perfect match. But, it reminded me that the table which is now throwing errors had recently been modified in the dbml file. Time to check the structure of the table throwing the error.
A new varchar field in the table was not marked as nullable in the dbml. This means to me that at some layer in the update process, null was being compared to empty string and .Net was unable to discern which row in the database was in need of an update.
I wanted to believe that since the primary key on the table was defined in the dbml the remainder of the table definition should have no impact on resolution of the row being updated... But apparently not in the use case I was fighting...
After a great presentation by Jason Sheehan at MDC about RestSharp, I decided to implement it.
RestSharp is a .Net framework for consuming restful data sources via either Json or XML.
My first step was to put together a Restful data source for RestSharp to consume. Staying entirely withing .Net, I decided to use Microsoft's oData implementation, built on System.Data.Services.DataServices. Natively, these support Json, or atom+pub xml. (XML with a few bells and whistles added on)
There are three main steps for creating an oData data source:
1) override CreateDSPMetaData
This is where the metadata data is returned. The meta data defines the structure of the data to return. The structure contains the relationships between data objects, along with what properties the objects expose. The meta data can and should be somehow cached so that the structure is not rebuild with every data request.
2) override CreateDataSource
The context contains the data the data source will publish. This method is the conduit which will populate the metadata objects to be returned to the requestor.
3) implement static InitializeService
At this point we can set up security, along with setting up properties of the web service (versioning, etc)
Here is a web service which publishes stock prices for various Products (stocks) in various Categories.
namespace RestService
{
public class RestServiceImpl : DSPDataService<DSPContext>
{
private static DSPContext _context;
private static DSPMetadata _metadata;
/// <summary>
/// Populate traversable data source
/// </summary>
/// <returns></returns>
protected override DSPContext CreateDataSource()
{
if (_context == null)
{
_context = new DSPContext();
Category utilities = new Category(0);
utilities.Name = "Electric";
Category financials = new Category(1);
financials.Name = "Financial";
IList products = _context.GetResourceSetEntities("Products");
Product electric = new Product(0, utilities);
electric.Name = "ABC Electric";
electric.Description = "Electric Utility";
electric.Price = 3.5;
products.Add(electric);
Product water = new Product(1, utilities);
water.Name = "XYZ Water";
water.Description = "Water Utility";
water.Price = 2.4;
products.Add(water);
Product banks = new Product(2, financials);
banks.Name = "FatCat Bank";
banks.Description = "A bank that's almost too big";
banks.Price = 19.9; // This will never get to the client
products.Add(banks);
IList categories = _context.GetResourceSetEntities("Categories");
categories.Add(utilities);
categories.Add(financials);
utilities.Products.Add(electric);
utilities.Products.Add(electric);
financials.Products.Add(banks);
}
return _context;
}
/// <summary>
/// Setup rules describing published data structure - relationships between data,
/// key field, other searchable fields, etc.
/// </summary>
/// <returns></returns>
protected override DSPMetadata CreateDSPMetadata()
{
if (_metadata == null)
{
_metadata = new DSPMetadata("DemoService", "DataServiceProviderDemo");
// Define entity type product
ResourceType product = _metadata.AddEntityType(typeof(Product), "Product");
_metadata.AddKeyProperty(product, "ProductID");
// Only add properties we wish to share with end users
_metadata.AddPrimitiveProperty(product, "Name");
_metadata.AddPrimitiveProperty(product, "Description");
EntityPropertyMappingAttribute att = new EntityPropertyMappingAttribute("Name",
SyndicationItemProperty.Title, SyndicationTextContentKind.Plaintext, true);
product.AddEntityPropertyMappingAttribute(att);
att = new EntityPropertyMappingAttribute("Description",
SyndicationItemProperty.Summary, SyndicationTextContentKind.Plaintext, true);
product.AddEntityPropertyMappingAttribute(att);
// Define products as a set of product entities
ResourceSet products = _metadata.AddResourceSet("Products", product);
// Define entity type category
ResourceType category = _metadata.AddEntityType(typeof(Category), "Category");
_metadata.AddKeyProperty(category, "CategoryID");
_metadata.AddPrimitiveProperty(category, "Name");
_metadata.AddPrimitiveProperty(category, "Description");
// Define categories as a set of category entities
ResourceSet categories = _metadata.AddResourceSet("Categories", category);
att = new EntityPropertyMappingAttribute("Name",
SyndicationItemProperty.Title, SyndicationTextContentKind.Plaintext, true);
category.AddEntityPropertyMappingAttribute(att);
att = new EntityPropertyMappingAttribute("Description",
SyndicationItemProperty.Summary, SyndicationTextContentKind.Plaintext, true);
category.AddEntityPropertyMappingAttribute(att);
// A product has a category, a category has products
_metadata.AddResourceReferenceProperty(product, "Category", categories);
_metadata.AddResourceSetReferenceProperty(category, "Products", products);
}
return _metadata;
}
/// <summary>
/// Based on the requesting user, can set up permissions to Read, Write, etc.
/// </summary>
/// <param name="config"></param>
public static void InitializeService(DataServiceConfiguration config)
{
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
config.DataServiceBehavior.AcceptProjectionRequests = true;
}
}
}
The objects prefixed with DSP come from the samples on the oData site: http://www.odata.org/developers
The products and categories objects are POCO business objects with no special modifiers.
Three main options are available for defining the MetaData of data sources in .Net:
1) Generate Entity Data model (Potentially directly from SQL Server database). This requires the least amount of manual interaction, and uses the edmx WYSIWYG editor to generate a data model. This can be directly tied to the SQL Server database and generated from the database if you want a data access layer tightly coupled with your database.
2) Object model decorations. If you already have a POCO data layer, you can decorate your objects with properties to statically inform the compiler how the objects are related. The disadvantage is there are now tags strewn about your business layer that need to be updated as the business rules change.
3) Programmatically construct metadata object. This is the object illustrated above in CreateDSPMetaData. This puts all relationship information into one central programmatic location. Here business rules are constructed when the DSPMetaData response object is returned.
Once you have your service up and running, RestSharp is designed for XML / Json, along with the native Microsoft library. There are currently some differences between how Jason made RestSharp expect XML with how atom+pub works, so I found better results currently with the Json implementation - modifying the RestSharp XML parser to make an atom+pub parser is fairly trivial though, so use what implementation works best for you.
I put together a sample console app which calls the RestSvcImpl.svc service defined above (and assumes it to be running on port 2000). I used both RestSharp as a client, and also the default Microsoft oData client tools.
namespace RestConsole
{
class Program
{
private static DataServiceContext _ctx;
private enum DemoType
{
Xml,
Json
}
static void Main(string[] args)
{
// Microsoft implementation
_ctx = new DataServiceContext(new System.Uri("http://localhost:2000/RestServiceImpl.svc"));
var msProducts = RunQuery<Product>("Products").ToList();
var msCategory = RunQuery<Category>("/Products(0)/Category").AsEnumerable().Single();
var msFilteredProducts = RunQuery<Product>("/Products?$filter=length(Name) ge 4").ToList();
// RestSharp implementation
DemoType demoType = DemoType.Json;
var client = new RestClient("http://localhost:2000/RestServiceImpl.svc");
client.ClearHandlers(); // Remove all available handlers
// Set up handler depending on what situation dictates
if (demoType == DemoType.Json)
client.AddHandler("application/json", new RestSharp.Deserializers.JsonDeserializer());
else if (demoType == DemoType.Xml)
{
client.AddHandler("application/atom+xml", new RestSharp.Deserializers.XmlDeserializer());
}
var request = new RestRequest();
if (demoType == DemoType.Json)
request.RootElement = "d"; // service root element for json
else if (demoType == DemoType.Xml)
{
request.XmlNamespace = "http://www.w3.org/2005/Atom";
}
// Return all products
request.Resource = "/Products?$orderby=Name";
RestResponse<List<Product>> productsResp = client.Execute<List<Product>>(request);
List<Product> products = productsResp.Data;
// Find category for product with ProductID = 1
request.Resource = string.Format("/Products(1)/Category");
RestResponse<Category> categoryResp = client.Execute<Category>(request);
Category category = categoryResp.Data;
// Specialized queries
request.Resource = string.Format("/Products?$filter=ProductID eq {0}", 1);
RestResponse<Product> productResp = client.Execute<Product>(request);
Product product = productResp.Data;
request.Resource = string.Format("/Products?$filter=Name eq '{0}'", "XYZ Water");
productResp = client.Execute<Product>(request);
product = productResp.Data;
}
private static IEnumerable<TElement> RunQuery<TElement>(string queryUri)
{
try
{
return _ctx.Execute<TElement>(new Uri(queryUri, UriKind.Relative));
}
catch (Exception ex)
{
throw ex;
}
}
}
}
Feel free to step through the code a few times and to attach a debugger to the service as well to see how and where the context and metadata objects are constructed and returned. Pay special attention to the response object being returned by the oData service - There are several properties of the RestRequest that can be used to help troubleshoot when the structure of the response is not exactly what would be expected.
There was a recent upgrade of the Oracle client on a production web server housing serveral app pools each containing multiple web sites. The upgrade consisted of adding the 11.2 client tools on the web server, but leaving the 9.2 and 10.2 versions alone.
After learning the upgrade occured, I updated Oracle.DataAccess.dll on my site to the 11.2 dll, and the web site ran smoothly.
The next day I discover the site is throwing an error: Oracle.DataAccess.Client.OracleException: The provider is not compatible with the version of Oracle Client
The App Pool will connect with an installed version of the Oracle client provider, and coordinate database interactions. If a request comes in for a web site configured using an earlier version of the provider, that is the version the App Pool will use for Oracle interactions. If a request is made for a site configured for a different version of the provider, the App Pool will see a conflict and throw the above error.
This web site: http://splinter.com.au/blog/?p=156 described the steps to grab the DLLs for a provider and bring them local (for Oracle 11.1), but even when I brought the DLLs for the Oracle Provider into the local bin directory, the App Pool would still determine the version of the oracle client and continue to throw the exception.
My only resolution for the above problem has been to have dedicated IIS app pools per version of Oracle client being leveraged.
In a ASP.Net web app I have a button that says "Process". It will spawn a new worker thread that will look at a network share and process some Excel files that have been produced by a third party system.
To access the share and to use the Excel COM interop, I am impersonating an elevated user account different than the authenticated user, using Windows Impersonation. I then generate an Excel COM object for each Excel file, and process the files available in the network share.
Today I noticed that when the number of Excel files in the process had grown to around 30, I suddenly am getting a permissions error:
Server Error in '/excel' Application.
Retrieving the COM class factory for component with CLSID {00024500-0000-0000-C000-000000000046} failed due to the following error: 80070005.
This error suggests that my current user does not have permission to open the COM object. True, I'm running an ASP.Net web app on Windows Server, so the thread would be the server level account running the w3wp, but as before, this is wrapped in code using Windows Impersonation, so I'm doing this as an administrator account.
Not sure what to think, my first wild guess is that somehow my impersonation context is expiring. As an expirement I make the code stop and restarting the impersonation between each file. The process now completes successfully. But now I notice that I have spawned a new EXCEL process for every different xlsx file that was processed.
After more research, the Excel App object is associated with the current user. It appears (?) that switching context so quickly is affected Window's ability to dispose of all of those Excel instances. More importantly: the real problem is that generating so many Excel App objects is having an effect on my impersonation, and that number of Excel App objects is affecting my ability to construct a new object and an impersonation context should never time out.
In the process file method, every file will construct a new Excel object, then dispose and garbage collect the Excel App object upon completion.
I replace the logic to share one common Excel App object between all of the Excel input files, and the problem is now resolved. My one impersonation object does not need to be re-created, and I only go through the effort of constructing one Excel App object which is leveraged to load the various Workbooks from.