As part of Processing Performance improvements, I was trying Processing ByTable option for Dimension instead of default ByAttribute option. I'm testing this on a Production like environment with 96 GB RAM, so memory shouldn't be a problem.
Before trying ByTable option, when I do a ProcessUpdate on a dimension with ByAttribute, it takes 30 minutes to finish. When i try ProcessUpdate with ByTable on the same dimension, the processing fails after 10 minutes with error:
File system error: A FileStore error from WriteFile occurred. Physical file: \\?\D:\SSAS\OLAP\Temp\MSMDCacheRowset_3016_b44_8mwces.tmp. Logical file: . ." Source="Microsoft SQL Server 2008 R2 Analysis Services" HelpFile="" />
Firstly, my question is about the path it shows. I have come across this error with SSAS\OLAP\Data folder location. Is it because of the fact that I'm using ByTable, it is creating the data store files in Temp folder first? When I check the folder structure on the server, Temp folder is empty. However Data folder has the updated files. So there is a bit of confusion here.
Assuming the usual location of Data folder for the data store files, I found one asstore file for an attribute which is around 3.5 GB. The Kstore and Khstore files for this attribute are around 1.5 GB each. The number of records in the dimension is 58160543 and the datatype of the attribute in question is Integer. So the maximum possible size is roughly 4 bytes * 58160543 which is less than 1 GB. So none of the files are nearing the 4GB limit on the data store for SQL Server 2008. Where exactly am I going wrong in my assumptions?
When ProcessUpdate with ByAttribute setting succeeds, I thought it is safe to imagine that the same operation with ByTable would be faster specially with memory not being a constraint.
I want to get to the root of this problem. Any advice would be appreciated.
Pavan