SQLServer

SSAS, MPP and PDW

James Rogers — Fri, 14 Oct 2011 00:01:32 GMT

I cut my DBA teeth on IBM's DB2 UDB EEE (enough acronyms) back in the 90's. I have always appreciated the scalability that is available with a MPP architecture. So a couple of years ago I was excited to hear that SQL Server was going to have a MPP architecture available. Now I am at SQLPass Summit 2011 and seeing the PDW (Parallel Data Warehouse) and the hardware offerings by Dell and HP to scale that up and out. However, I was disappointed to learn that this technology does not extend to Analysis Services.

I spend a great deal of my time tuning SSAS and can say that it scales very well as is and there is a plethora of information out there to facilitate those efforts. However, let's dream a little here. Imagine a multi-node/single-instance/shared-nothing installation of SSAS. You can distribute/partition the cubes across the nodes and then within the nodes in a manner that allows you to isolate hot-spots of data on their own hardware so that data can stay cached. Meanwhile, other queries on data that is less-often accessed or more ad-hoc in nature can process with resources allocated on separate nodes for that and the majority of queries do not suffer as much from cache turnover and the overall system load. Ahhh, it is good to dream. We could take that a step further and wish that would could mix Vertipaq storage for that more frequently used/"hot" data node and traditional Analysis Services storage for the other data where aggregations may have more value in helping the queries. Additionally, processing of mining models could be further isolated from ongoing activities and processing activities could take advantage of the MPP architecture and resulting parallelism to allow for more processing in shorter time than on a similar, single box solution. Could even go further and dedicate a node to ROLAP partitions for more real-time requirements while limiting the burden of such queries on the overall system performance.

I realize that MS probably doesn't have much of a target market for such dreams but it would be cool nonetheless and provide some real scalability options for big data analytics implementations. Like always, the devil is in the details. OK - back to reality.

Analysis Services (SSAS) - Unexpected Internal Error when processing (ProcessUpdate). Workaround/Resolution

James Rogers — Mon, 20 Dec 2010 17:51:05 GMT

/* EDIT - This problem has been fixed in the latest SQL Server 2008 R2 Cumulative Update package. It was identified that changing the aggregation design without reprocessing the aggregations prior to a ProcessUpdate on the dimensions causes this problem and it is a bug */

Many implementations require the use of ProcessUpdate to support Type 1 slowly changing dimensions. ProcessUpdate drops all of the affected indexes and aggregations in partitions affected by data that changes in the Dimension on which the ProcessUpdate is being performed. Twice now I have had situations where the processing fails with "Internal error: An unexpected exception occurred." Any subsequent ProcessUpdate processing will also fail with the same error. In talking with Microsoft the issue is corrupt indexes for the Dimension(s) being processed in the partitions of the affected measure group. I cannot guarantee that the following will correct your problem but it did in my case and saved us quite a bit of down time.

Workaround: ProcessIndexes on the entire cube that is being processed and throwing the error. This corrected the problem on both 2008 and 2008 R2.

Pros:

Does not require a complete rebuild of the data (ProcessFull) for either the Dimension or Cube.
User access can continue while this ProcessIndexes in underway.

Cons:

Can take a long time, especially on large cubes with many partitions, dimensions and/or aggregations.
Query Performance is usually severely impacted due to the memory and CPU requirements for Aggregation and Index building

<DatabaseID>MyDatabase</DatabaseID>

<CubeID>MyCube</CubeID>

</Object>

<Type>ProcessIndexes</Type>

<WriteBackTableCreation>UseExisting</WriteBackTableCreation>

</Process>

</Parallel>

</Batch>

The cube where the corruption exists can be found by having Profiler running while the ProcessUpdate is executing. The first partition that displays the "The Job has ended in failure." message in the TextData column will be part of the cube/measuregroup that has the corruption. You can try to run ProcessIndexes on just that measure group. This may correct the problem and save additional time if you have other large measure groups in the cube that are not affected by the corruption.

Remember to execute your normal ProcessUpdate batch after the successful completion of the ProcessIndexes. The ProcessIndexes does not pick up data changes.

Things that did not work:

ProcessClearIndexes - why this doesn't work and ProcessIndexes does is unclear at this point.
ProcessFull on the partition in question. In my latest case, this would clear up the problem for that partition. However, the next partition the ProcessUpdate touched that had data in it would generate and error. This leads me to believe the corruption problem will exist in all partitions in the affected measure group that have data in them.

NOTE: I experience this problem in both a SQL 2008 and SQL 2008 R2 Analysis Services environment, on separate built from the same relational database. This leads me to believe that some data condition in the tables used for the Dimension processing caused the corruption since the two environments were on physically separate hardware. I am waiting on Microsoft to analyze the dumps to give us more insight into what actually caused the corruption and will update this post accordingly.

SQL Azure shows promise for small data applications - not there for BI yet.

James Rogers — Thu, 05 Nov 2009 00:49:22 GMT

I am currently sitting in a seminar on BI in the Cloud by John Welch at the PASS Summit 2009. SQL Server Azure looks promising for a variety of applications. Currently it only supports SQL Server relational database services but future plans to include the BI stack of SQL Server.

I have yet to see how loading of data is handled and security is handled but it seems to have a lot of potential for small data sets (< 10 GB).

Couple of points:

Apparently data manipulation is slower than normal SQL Server.
Queries are limited to 5 minutes
Max database size is 10GB
Bulk Insert not available yet but available in an upcoming CTP
No control over physical database implementation (files,filegroups,partitions, etc…)
No Windows auth supported