Dan English's BI Blog

Welcome to my random thoughts in regards to Business Intelligence, databases, and other technologies

Archive for April, 2008

BIDS Helper on CodePlex.com

Posted by denglishbi on April 26, 2008

I blogged about the new release back in December here when version 1.2.0.1 came out which fixed a few issues with the SSAS buttons in the calculation tab and with the SSIS highlighter.  In talking with colleagues and clients recently though it seems like quite a few people are surprised when I talk about BIDS Helper.  I thought everyone already knew about it or was using it, but I guess not.  I really feel that this is an excellent add-in for anyone doing SSAS or SSIS development that I had to post another blog posting.  I know that my blog is starting to get more coverage thanks to the BI Blog aggregator http://biglogs.com and now that my blog is mirrored on the Magenic blog site at http://blog.magenic.com/blogs I wanted to post another entry promoting the BIDS Helper which is available here.

So what is so cool about BIDS Helper and why would you want to use it?  Well the main features that I like are the options to deploy the MDX script in the calculations tab of the cube, aggregation manager, update estimated counts, column usage reports, expression and configuration highlighter, expression list, and smart diff (all of the features are cool and helpful, but these are some of the key ones I use).  If you go to the site on CodePlex you can get detailed information about each feature with screenshots. 

It appears that this add-in currently only works with the English version of BIDS and that it probably only works with 2005 and not 2008 versions of Visual Studio.  I haven’t actually tested it in VS 2008 yet, but according to some comments on the site there appears to be some issues.

Here is the complete list of features that are currently included with the BIDS Helper along with a picture from the site that highlights a few of the features:

Features

 

 image

I did take a quick peek at the Source Code section for this Project and it appears that version 1.3.0.0 should be available anytime now.  Some of the new features according to the comments are the ability to sort the SSIS packages by name (like you can do in SSAS), VS 2008 support, SQL Sever 2008 support, smart diff for SSAS objects, and more performance improvements Hot.  This is cool stuff and you got to thank Darren Gosbell, Greg Galloway, and John Welch (hopefully I got all of the names right) for making this available.  Feel free to post an item on the discussion section for anything that you would like to see added because most of the new features being added are taken directly from this section here.

Posted in Downloads | Tagged: , | Leave a Comment »

Microsoft Business Intelligence VPC Release 6

Posted by denglishbi on April 26, 2008

Yesterday I received an email from my colleague, Kory, asking me if I had heard of the new BI VPC version 6 (April 2008 release) that was available.  I stated that the only one that I knew of was version 5.1 which I posted on my blog back in December here.  I quickly queried the Microsoft Download Center here and I didn’t see a new version available.  I asked him where he saw this mentioned and he pointed me to a new BI blog that I had not seen yet by Peter Koller over in Norway here.  Now I had to know where Peter discovered this so I left him a comment and he pointed me to the hidden gem buried on Microsoft’s site here (look down in the lower right for the download links – 7 files).

So what is different.  Version 6 includes new demos and presenter scripts for different verticals.  When you extract the files it creates an additional demos folder which now contains all of the scripts so you don’t need to try and pull them off the VPC at all.  It still has all of the same programs installed and configured which is very nice Smile.  Here is everything it contains and I highlighted the new items (and this VPC will expire on 8/17/2009 – maybe just in time for PPS v2).

 image

I didn’t see a readme file associated with this, so if you are looking for the password it is the typical Microsoft demo one and the same one that was used on the last VPC, pass@word1.  So let the downloads begin!  Enjoy the VPC and I got to thank Kory and Peter for pointing this out and sharing this information with the community, thanks guys!

If for some reason the link above does not work for you then here is the direct links to the downloads:

Virtual PC

Instruktioner

Business Intelligence VPC Release 6.part01.exe

Business Intelligence VPC Release 6.part02.rar

Business Intelligence VPC Release 6.part03.rar

Business Intelligence VPC Release 6.part04.rar

Business Intelligence VPC Release 6.part05.rar

Business Intelligence VPC Release 6.part06.rar

Business Intelligence VPC Release 6.part07.rar

 

Posted in Downloads | Leave a Comment »

Did you know? SQL Server 2008 includes GROUPING SETS

Posted by denglishbi on April 24, 2008

SQL Server 2008 includes GROUPING SETS:

Grouping Sets is an extension to the GROUP BY clause that lets users define multiple groupings in the same query. Grouping Sets produces a single result set that is equivalent to a UNION ALL of differently grouped rows, making aggregation querying and reporting easier and faster.

Read more about this in the SQL Server 2008 BOL ‘GROUPING SETS’.

Check out a screencast about the new GROUPING SETS feature in SQL Server 2008 here.

Did you know? SQL Server 2008 includes GROUPING SETS from Dan English on Vimeo.

Posted in SQL Server | Tagged: , , | Leave a Comment »

SSAS MDX Round = Banker’s Rounding

Posted by denglishbi on April 19, 2008

A couple of weeks ago I was working on a project where I had to implement some Ranking calculations within an Analysis Service (SSAS) cube.  I went ahead and added in the calculations and then the client came back to me and stated that there was some additional business logic that we needed to implement.  At different levels of reporting they utilize some rounding into the calculation that is used to determine the Ranking.  When viewing the report at a store level then we need to round the calculation to two decimal places and at all other levels the value will be rounded to four decimal places to be used in the Ranking function.  I thought to myself ‘no problem’ and I went ahead and added in this additional logic and put the Round function into place to handle this logic.  Done.

Well was I wrong.  The client came back to me and said that stores were not being ranked properly and why if these had the same calculation score, 93%, did they not have the same rank.  I was like ‘what?  all i did was add the business logic you requested’.  So I went ahead and started digging into the issue and this opened up a whole can of worms that took me a couple of days to sort out and it was a nightmare — rounding is just plain bad.  What we were seeing on the report was a score of 93% and two scores would have values of say 92.75% and 92.50%.  Now back from my school days using the basic arithmetic round that we all used we would say that these should both be rounded up to 93%, well that ‘depends’.  It turns out that there are quite a few methods of rounding and unfortunately the Microsoft products all differ in how they use the round function and I found a good article explaining that here.  It turns out that in this case Banker’s Rounding was implemented where if the value is at say .5 it will simply round to the nearest even number.  And sure enough this is what was happening; here are some examples that show this (replace cube name with a valid cube name):

–this does not round 92.5 up to 93 as expected

with member measures.test1 as round(.923,2)

     member measures.test2 as round(.925,2)

     member measures.test3 as round(.927,2)

select {measures.test1, measures.test2, measures.test3} on columns

from [cube name]

This was not what we wanted, so now I was on a quest to resolve this.  I thought I could just try and utilize the vbamdx.round call, but that returned the same thing.  I went ahead and tried using the system.math.round call in .NET code to see what that returned to see if I could possibly just implement an SSAS stored procedure (functions are what I call them), but same thing.  I opened up my trusty Excel and tried the round function in there and it worked the way I expected it to.  I was like it is about time…so I thought I would simple use the excelmdx.round in my calculation, but it turns out you actually need to install Excel on the server (found reference to this in the MSDN forums here) for this to work and it will fire up an instance of the excel object when you use this.  So this was not a solution either since we were not going to do that.  I found another solution in the MSDN forums here that stated that you could just use the following logic -int(-(VALUE*100)) and sure it works for my .9250 value, but this logic is basically just returning the CEILING and this is not what I wanted for all of my values (like .9210 would display as 93 — not what we wanted).  What I ended up doing was implementing some crazy CASE logic that evaluated the value to determine if the last numeric value on the right was a 5 then use the CEILING function that I just listed, otherwise just go with the regular round function.

So now I was done, right?  Wrong.  Now we were still getting a few stores that were still not lining up where the report would show 93%, but what was being used by the Ranking was still 92%, so now what was the problem.  It turns out the calculation was returning .9250, but it was actually .92499999, so why was the calculation returning .9250?  I didn’t have any type of formatting in the calculation, this was just a straight calculation returning the underlying value.  This was driving me completely insane.  How did I go about resolving this additional rounding issue, good question.  I ended up doing the following: int(value*100000)/100000 and using this value instead of just value.

So now I was done, right? Wrong again.  Now we were still seeing some incorrect values.  This is what we ended up seeing and shows the different values returned by int and cint.

      value = .57549

                Incorrect calc: int(round(value),2)*100) = 57

                Fixed calc: cint(round(value),2)*100) = 58

I went ahead and changed my references of int to cint and finally I had victory!  I was actually done solving this crazy rounding issue.  It wouldn’t have been that bad if we were just rounding and displaying values; this could have basically been handled by using some formatting in the calculation, but when you need to actually use the result in a Ranking function it is a completely different story.  I am really hoping that there was an easier solution for this, but being under the gun and needing this implemented by the deadline this is what I came up with.  I was able to finally get a good night sleep.

UPDATE (10/12/2009): Additional SSAS Rounding information and workaround posted here – Custom Rounding and Truncation of Numbers in MDX.

 

Posted in Analysis Services | Tagged: | 4 Comments »

Did you know? SQL Server 2008 includes Cache Transform screencast

Posted by denglishbi on April 15, 2008

SQL Server 2008 includes Cache Transform:

The Cache Transform transformation writes data from a connected data source in the data flow to a Cache connection manager.  The cache can be populated from text files, XML files, Excel files, etc.; you are not limited to only populating the cache from an OLE DB source.  The Lookup transformation in a package performs lookups on the data using the Cache connection manager.

You can configure the Cache connection manager to save the data to a cache file (.caw).  The cache file can be populated once and reused throughout the ETL process within multiple Lookup transformations for performing surrogate key lookups and populating the data warehouse.

Read more about this in the SQL Server 2008 BOL ‘Cache Transform’.

Check out a screencast about the new cache transform transformation feature in SQL Server 2008  here.

Did you know? SQL Server 2008 includes Cache Transform from Dan English on Vimeo.

If you are interested in additional information in regards to this topic check out these other blog postings and sites: 

You can also check out a video on YouTube done by Jamie Thomson in regards to this new transformation here.  The last time I viewed it there wasn’t any audio, just video.

You can download the SSIS samples from CodePlex and take a look at a sample utilizing the cache transform here.

Posted in Integration Services | Tagged: , , | 2 Comments »

Did you know? SQL Server 2008 includes Data Profiling Task screencast

Posted by denglishbi on April 6, 2008

SQL Server 2008 includes Data Profiling Task:

The Data Profiling task is a new task in the Integration Services toolbox. You can use this task inside an Integration Services package to profile data that is stored in SQL Server. The information provided by the profile helps you identify potential problems with data quality. The Data Profiling task provides profiles that help identify data quality problems within individual columns and with column relationships:

Profiles that help identify problems within individual columns

· The distribution of lengths in the column values.

· The percentage of null values.

· The distribution of values in the column.

· Column statistics for numeric columns.

· Regular expressions that match string columns.

Profiles that help identify problems with column relationships

· Candidate key columns.

· Functional dependencies between columns.

· The inclusion of the set of values in one column in the set of values in another column.

Read more about this in the SQL Server 2008 BOL ‘Data Profiling Task’.

Check out a screencast about the new data profiling task feature in SQL Server 2008  here.

Did you know? SQL Server 2008 includes Data Profiling Task from Dan English on Vimeo.

If you are interested in additional information in regards to this topic check out these other blog postings:

http://www.sqlskills.com/blogs/stacia/2007/11/27/SQLServer2008DataProfilingTaskTheQuickVersion.aspx

OR

SSIS: Data Profiling Task: Part 1 – Introduction
Data Profiling Task: Part 2 – Column Length Distribution
Data Profiling Task: Part 3 – Column Null Ratio
Data Profiling Task: Part 4 – Column Pattern
Data Profiling Task: Part 5 – Column Statistics

Data Profiling Task: Part 6 – Column Value Distribution
Data Profiling Task: Part 7 – Functional Dependency
Data Profiling Task: Part 8 – Candidate Key
Data Profiling Task: Part 9 – Value Inclusion

SSIS: Data Profiling Task: Part 10 – Parsing the output

and if you want to know how to dynamically profile tables check out this blog posting which includes the code here.

Did you know? SQL Server 2008 includes Data Profiling Task from Dan English on Vimeo.

If you are interested in additional information in regards to this topic check out these other blog postings:

http://www.sqlskills.com/blogs/stacia/2007/11/27/SQLServer2008DataProfilingTaskTheQuickVersion.aspx

OR

SSIS: Data Profiling Task: Part 1 – Introduction
Data Profiling Task: Part 2 – Column Length Distribution
Data Profiling Task: Part 3 – Column Null Ratio
Data Profiling Task: Part 4 – Column Pattern
Data Profiling Task: Part 5 – Column Statistics

Data Profiling Task: Part 6 – Column Value Distribution
Data Profiling Task: Part 7 – Functional Dependency
Data Profiling Task: Part 8 – Candidate Key
Data Profiling Task: Part 9 – Value Inclusion

SSIS: Data Profiling Task: Part 10 – Parsing the output

and if you want to know how to dynamically profile tables check out this blog posting which includes the code here.

Posted in Integration Services | Tagged: , , | 1 Comment »

Did you know? SQL Server 2008 includes Intellisense screencast

Posted by denglishbi on April 6, 2008

A new thing we are trying internally at work is sending out weekly ‘Did you know?’ (DYN) emails about new features that are included in SQL Server 2008.  We are trying to spread awareness and excitement about the new version of SQL Server 2008.  The emails include information about a new feature, links to the topic in BOL, a internal discuss in regards to the topic, and SQL Server 2008 information (main page site, trial download site, learning portal, webcasts, and virtual labs).  The other item that we are including is a screencast of the new feature.  So far I have done two of the DYN topics and I thought I would start sharing them with the rest of the community.  Here is the screencast for Intellisense and I will post the other screencast on Data Profiling task next.

SQL Server 2008 includes T-SQL IntelliSense:
Transact-SQL IntelliSense provides intelligent aids for Transact SQL scripting that make language references easily accessible for database developers. When coding, you do not need to leave the Database Query Editor to perform searches on T-SQL language elements or your database metadata.  You can keep your context, find the information you need, insert T-SQL language elements directly into your code, and even have IntelliSense complete your typing for you. This can speed up software development by reducing the amount of keyboard input required and minimize references to external documentation.

Read more about this in the SQL Server 2008 BOL ‘Using Intellisense’.

Check out a screencast about the new intellisense feature in SQL Server 2008 here.  

Did you know? SQL Server 2008 includes Intellisense from Dan English on Vimeo.

Posted in SQL Server | Tagged: , , | Leave a Comment »

Heroes Happen {Here} Software

Posted by denglishbi on April 5, 2008

Just another thing in regards to attending one of the local launch events, free softwareSmile  You receive a sticker when you get checked-in at the event to get the software at the end of the event.  The software bundle included Windows Vista Ultimate with SP1 (with a trial version of Windows Live OneCare), Windows Server 2008 Enterprise (32 and 64 bit versions – one year evaluation license included), SQL Server 2008 Developer Nov 2007 CTP (32 and 64 bit versions – with a voucher for a evaluation copy of SQL Server 2008 Standard when released), Visual Studio 2008 Standard with MSDN library, Windows Mobile 6 Developer Resource Kit (Dec 2007), and Microsoft Forefront and System Center (limited-time trial software).  Wow!  No T-Shirt for the launch, but talk about a fabulous software bundle for just attending the launch.  Outstanding!

Don’t miss a chance to attend if you haven’t already missed it in your local area — Heroes Happen {Here} Registration.

One other thing, stop by the Microsoft Business Intelligence booth and pickup a copy of the Microsoft BI Resource Kit if you haven’t gotten one already.  This was release at the Microsoft BI Conference last year, but it is loaded with lots of useful white papers and webcasts.

Posted in Personal Comments | Leave a Comment »

Hereos Happen {Here} Hands on Lab Manuals

Posted by denglishbi on April 5, 2008

If you went to one of the launches there was a hands on lab setup with some great exercises that you could take to take a look at some of the features that are available in Windows Server 2008, Visual Studio 2008, and SQL Server 2008.  I did some of the PowerShell labs for Windows Server 2008 (a few minor issues in the labs, but nothing that I couldn’t figure out) and Change Data Capture (CDC) and Table Partitioning (not a new feature in 2008, exists in 2005 – Enterprise Edition) in SQL Server 2008.

If you haven’t attended a launch yet I would highly recommend checking out the virtual lab area, but if you don’t get a chance or missed out here is the link to the site that has all of the lab manuals you can download.

Heroes Happen {Here} Hands on Lab Manuals

Posted in Training | Tagged: | Leave a Comment »

TechNet BI Virtual Labs

Posted by denglishbi on April 5, 2008

UPDATED (11/19/2010): Fixed some of the links for the labs, still tracking down some if they still exist.  Might want to check out this site as well for SQL Server 2008 Virtual Labs – http://www.microsoft.com/events/series/technetsqlserver2008.aspx?tab=virtuallabs.

Just wanted to point out the Business Intelligence virtual labs that are available on the TechNet web site.  There are three labs in regards to PerformancePoint server tooOpen-mouthed

Business Intelligence

 

Don’t forget to check out the ones in regards to SQL Server 2008 and get ready for the new product release and take the new version for a test drive.  There are two locations for these virtual labs, there are some on MSDN and some on TechNet.

Microsoft SQL Server 2008 Virtual Labs

TechNet Virtual Labs: SQL Server 2008

Posted in Training | Tagged: | 1 Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 1,758 other followers