SQLRally Voting Opens to the Community

PASS SQLRally 2012 voting is now open! There’s tons of great sessions out there and many of them have been submitted by some of my good friends here at Pragmatic Works. While voting look for great sessions presented by Pragmatic Works experts like

  • Adam Jorgensen
  • Chris Albrektson
  • Gareth Swanepol
  • Dan Clark
  • Brad Schacht
  • and me, too!

Make sure you head over to SQLRally.com and cast your vote! You can vote for as many sessions as you want but you can only vote once, so make your selections wisely! Yours truly has also submitted a few sessions that are currently up for vote! Here are my sessions up for vote and their descriptions.

Zero to Cube – Fast Track to Analytics
This session has been updated for SQL Server 2012. It is consistently a top session from previous events like SQLRally and PASS Summit. Come interact with your speaker and audience like you never have before. This session is 100% demo, packed full of fun and will get you up and running on Analysis Services 2008R2 and 2012 right away!

SSIS For Your Data Warehouse
The ability of SQL Server Integration Services to easily bring together multiple data sources makes it the perfect ETL tool for loading your data warehouse. Loading your data warehouse quickly and efficiently, however, is not always easy. In this session, we’ll explore some advanced techniques for loading your data warehouse and help you overcome some of the learning curve associated with SSIS. The topics discussed will include handling late arriving facts, loading slowly changing dimensions efficiently, and more.

SSIS Unleashed: Expressions and Variables Edition
One of the most powerful aspects of SSIS is the ability to create dynamic and powerful packages utilizing package variables and the SSIS expression language. Unfortunately, the SSIS expression language is also one of the more difficult features to learn regarding SSIS. In this session we will walk through the syntax of the SSIS expression language, common uses for SSIS expressions, as well as some more advanced ways to fully realize the power of SSIS.

Tackling the SSRS Expression Language
The SSRS expression language is one of the most powerful pieces when it comes to creating dynamic reports. Unfortunately, for reporting developers the SSRS expression language is also one of the more difficult features to use regarding SSRS. During this session we will walk through the syntax of the SSRS expression language, how and where to use the expressions, as well as some advanced ways to truly reveal the true power of your reports.

Now that you’re informed, go cast your vote!

BI Documenter 4.2 Released

I’ve been meaning to blog this for a while but since I’ve been on the road for the past month I’ve been pretty busy. This past January, BI Documenter version 4.2 was released by Pragmatic Works. With it comes some pretty cool upgrades and some new functionality.

One of the biggest improvements was the upgrade to the SQL Server Instance Snapshot functionality. It is now possible to include in your documentation Backup Devices, SQL Agent Jobs, and Server Configurations. Personally, I’ve been waiting for the ability to document SQL Agent Jobs so I’m very excited about this release. There’s also many other improvements to the SQL Server Instance Snapshot functionality, so head over to PragmaticWorks.com to check out the other improvements.

Theres also a couple new enhancements that are really nice. Now you have the option to output your documentation in MS Help 2 format. Also, BI Documenter can now document Analysis Services Mining Structures.

With the ability to document SQL Servers, SSIS packages, SSRS reports, and SSAS cubes, its certainly the most encompassing SQL documentation tool out there. If you’re searching for a one stop documentation tool for your SQL Server environment, definitely check out BI Documenter.

BI xPress Version 3.2.1 Released

Earlier this month on the 14th, version 3.2.1 of BI xPress was released. With this release comes a few nice improvements that increase the ease of use.

One of the major improvements with this release (and to be honest its about time) is the ability to view packages that have been stopped by some external source (i.e. the user, the operating system, etc.). The stopped packages will now appear highlighted in blue in the Package Execution viewer. Previously it was impossible to tell which packages had been stopped by the end user, but with this improvement it’s not possible to tell which package has been manually stopped by the end user.

Packages that have not started execution are white, packages currently executing appear in yellow, packages that have successfully executed appear in green, packages that have failed are red, and packages that have been stopped are now blue.

One of the other nice improvements is the ability to detect invalid file paths for configuration files. If the file path for a configuration file cannot be found, the configuration will appear underlined and highlighted in red in the Configuration Manager.

There’s a host of other improvements and bug fixes in this release. To read the full release notes, go here. To download the trial version of BI xPress, head over to PragmaticWorks.com.

Adventure Works Sample Databases for SQL 2012 RC0 Released

Early last week Microsoft released the Adventure Works samples for SQL Server 2012 RC0. Head over here to get the download. These sample databases only work with SQL Server 2012 RC0.

I love using these sample databases in my blog posts since they’re available to download for free and it makes it easy for those trying to follow along. The Adventure Works samples are also great for trying to recreate difficult issues my clients may be facing so I can have a better understanding of how to tackle the problem. If you’re serious about learning MS BI and SQL Server, you should definitely have some kind of sample database to play around with and test, so why not the AW sample since everyone has access to it.

The AW samples released for SQL Server 2012 RC0 use the AdventureWorks2008R2 and AdventureWorksDWDenali databases. There are no schema or data changes for in this release of the samples, except for additional date data in the DimDate table and an extra fact table, FactProductInventory, which contains inventory table. Download the sample databases here.

What #SQLFamily Means to Me

If you haven’t heard about some of the great things my company, Pragmatic Works, is doing to give back to our veterans integrating back into civilian life, check out what Microsoft is doing to get involved with this. In light of this, Microsoft wants to hear more about #SQLFamily and other amazing instances where someone in the SQL community has lent a helping hand. Microsoft has pledged to donate $50 for the first 400 stories submitted illustrating #SQLFamily. I thought it’d be appropriate, therefore, for me to share what #SQLFamily means to me. But first let me share how I ended up as a business intelligence consultant.

In 2008 I was loading trucks at UPS and had been for the previous six years as I progressed through college. I wasn’t happy there but it had great benefits so I was content to stick out and try and make something happen. I’ve always believed that a person can make anything happen if that person has the desire, the drive, and the determination, so I kept my head down and worked hard while hoping that some kind of management position would open up and give me the break I’d been looking for.

Then in August that year I learned through a buddy from high school, Devin Knight (blog | twitter), that his brother, Brian Knight (blog | twitter), was offering free training in something called “SQL”. Devin gave me a preview of the language and it seemed straight forward enough. I told Devin I was interested.

Devin put me in touch with Brian, whom I’d met a couple times before and we scheduled some time for me to start my training learning SQL and SQL Server Reporting Services. I went through three days of training with Brian and Mike Davis (blog | twitter) and learned about Select statements, data sources, reports, and tablixes. It was a lot to take in in just three days time, but I did my best. Brian and Mike were patient with me and answered the many questions I had.

Then one day I was driving home from work and Brian called me up. He was offering me a job as a Junior Business Intelligence Consultant. He said he saw a lot of upside in me. Brian wanted to take me in, mentor me, and train me. And as they say, the rest is history.

Since that day my life has never been the same. I won’t ever be able to thank Brian enough for giving me my shot. Because of Brian’s generosity, optimism, and faith in me, my life has changed for the better. And it’s because of Brian’s tremendous example that you can find me speaking at SQL Saturday and Code Camp events or answering questions on my blogs or the forums of BIDN.com. If I can have a fraction of the impact on someone else that Brian had in my life, then maybe in that way I can say thanks.

So that’s what #SQLFamily means to me. It means helping others because someone somewhere at some point gave you a helping hand. Whether it was a coworker who answered a question for you or someone who proofread your resume, take the time to say thanks and pay it forward to someone else who just needs a little faith, optimism, and a helping hand.

Find SSIS Variable Dependencies with BI xPress

One of the difficult aspects of SSIS package development is keeping track of where a variable has been used in an expression or whether a variable is being used at all. If you’re developing your packages with BI xPress, you can easily discover where your variables are being used if they’re being used at all.

Open your package in BIDS, and go to the Variables windows. Highlight the variables you want to check for dependencies and then click the Scan Variable Dependency icon.

1 Scan for variable dependencies icon

This will open the Variable Scanning Options. You can specify to scan all variables, which could take a considerable amount of time. In my case, I’m going to choose to scan for only the variables I have selected.

2 scan for selected variables

When the Variable Usage windows opens up, click the Scan all objects for variable usage button at the bottom. This is going to actually search our package to discover if and where the variables could be used.

3 View variable dependencies

Once the scan is complete, we can see on the right side of the window the tasks where our variables are used. In my case, my selected variable is used in an Execute SQL Task and a Script Task. If I highlight the Script Task, I can even view the script and see where the variable is used.

Any variables that are not being used in the package will show up under the Unused Variables node. At the bottom of the window you’ll notice that we have the option to either rename or delete these unused variables.

Head to PragmaticWorks.com to checkout BI xPress and download the free trial version.

Creating Named Sets In Your Cube

Named sets are simply MDX expressions defined with an alias that return a set of members. If you find yourself writing complex MDX expressions to return a set of members often or if you have a commonly used expression, consider creating a named set. This will prevent you from having to duplicate your work.

To create a named set, open your SSAS project and head to the Calculations tabs. Find the icon with the curly brackets and click it.

1 Click icon

Give your named set a name. For this example, I’m using the Adventure Works 2008 R2 cube and I’m creating a named set to return the top ten products with the highest Internet Sales Amount.

The MDX to return the top ten products with the highest Internet Sales Amount is very simple. We will use the TopCount function, which accepts three arguments.

The first argument is the dimension attribute we wish to return. The second argument is how many members will be returned in the set, and the last argument is the measure we would like to use to rank the members returned.

If I wanted to create a named set to return the Bottom 10 Products with the lowest Internet Sales Amount, I could write the same expression seen below except in place of the TopCount function I would use the BottomCount function.

1a Create Top 10 Count

When we create our named set, we can specify it as a Dynamic Named Set or a Static Named Set. A Dynamic Named Set respects the context of the subcube and the where clause and is evaluated at the time the query is executed. A Static Named Set is evaluated at the time the cube is processed and will not respect any subcube context and slicers in your where clause.

Here is the calculation script for the named set:

CREATE DYNAMIC SET CURRENTCUBE.[Top 10 Products]
AS
    TopCount
    (
    [Product].[Product].Members,
    10,
    [Measures].[Internet Sales Amount]
    )
     ;

The next time we process our cube, our named set will be available for use in our calculations and reports.

To use our named set in an MDX query, all I have to do is use the Named Set’s alias, [Top 10 Products].

Named Sets are powerful and very useful, but they can sometimes take a long time to write, especially if the expression is complex or if you are new to MDX. That’s where BI xpress steps in. BI xPress is a powerful tool used to speed up and augment SSAS and SSIS development. BI xPress also recently won the Gold Editors Choice Award from SQL Server Magazine.

To open the BI xPress Calculation Builder, open your SSAS project in BIDS, navigate to the Calculations tab and click the BI xPress Calculation Builder icon.

2 Click BI xPress icon

This will open the MDX Calculation Builder Wizard. Select the calculation or named set you would like to build. Scroll to the bottom to find the Sets templates. I’m selecting the Top 10 Count template.

3 Select Top 10 Count BI xPress

The first step is to select the attribute that you would like returned by the set. As before, I’m selecting the Product attribute of the Product dimension.

4 Select attribute

Then select the measure that you want to use to rank the selected attribute. I’m selecting the Internet Sales Amount measure.

4 Select measure

After I click next and give my calculation a name, the named set has been added to my calculation script in my cube.

To download the free trial of BI xPress, head over to PragmaticWorks.com and check out BI xPress. You won’t regret it after you see how much time it will save you with your SSAS and SSIS development.

BI xPress wins “Best Business Intelligence and Reporting Tool” from SQL Server Magazine

Last week it was announced that Pragmatic Works’ tool, BI xPress, won the SQL Server Magazine Gold Editors Choice Award for Best Business Intelligence and Reporting Tool and the Silver Community Choice Award. 

BI xPress allows for real time monitoring of all your SSIS packages executing on your servers. The way that BI xPress can allow you to monitor your SSIS package in real time is by applying a standardize auditing framework across all of your SSIS package using the BI xPress Auditing Framework Wizard. The Auditing Framework captures errors, warnings, and runtime details to create an execution log that can viewed in real time. This information makes it very easy to debug and track down problems with your SSIS packages.

So lets walk through applying the Auditing Framework to several packages a coworker developed to load a data warehouse.

To apply the Auditing Framework, I will select my packages in the Solution Explorer, right-click one of the packages and select Add/Remove Auditing Framework (BI xPress).

1 Select Packages

After selecting the packages, we need to set up the auditing database where all our package’s execution information will be stored. I can also choose to generate an configuration file for the connection to the auditing database.

3 Connection Information

On the next tab we select our logging options. We can enable real time monitoring, variable change tracking, row counts, connection information, and even SQL statements.

4 Logging Options

If we have any user variables we wish to track, we can specify those on the Advanced User Defined Logging tab.

5 Advanced User Defined Logging

After clicking Next we then applying the Auditing Framework.

7 Processing Summary

With the Auditing Framework applied, we can see the Row Count Transforms that are added to our Data Flow Tasks to track the rows from sources and into destinations.

8 Row Counts

Scripts tasks have also been added to Event Handlers to write to our Auditing Database when the packages execute or if an error or warning occurs.

9 Event Handler Script Tasks

With the Auditing Framework applied, we can now watch our packages execute in real time on the server.

10 Real Time Monitoring

As powerful as the Auditing Frame is, it’s only one of the features of BI xPress. Head over to PragmaticWorks.com to read more about the other features of BI xPress and download the trial version.

Ten MDX Calculations For Your Cube

I get lots of questions about how to implement various types of calculations in a cube so I figured I’d cover some of the more commonly used calculations I run into on a regular basis. All of these example calculations I’ll be providing were written against the Adventure Works 2008 R2 example cube but these should still work against later versions of the AW cube.

10. YTD, QTD, and MTD Calculations

These kinds of calculations are pretty common and I see these in a lot of cubes. They’re pretty easy to wire up since there are only a couple simple MDX functions necessary to make this work.

CREATE
MEMBER CurrentCube.[Measures].[YTD Internet Sales Amount] AS
Aggregate
(
PeriodsToDate
(
[Date].[Calendar].[Calendar Year]
,[Date].[Calendar].CurrentMember
)
,[Measures].[Internet Sales Amount]
)
,FORMAT_STRING = “Currency”
,VISIBLE = 1
,DISPLAY_FOLDER = ’10 MDX Calculations’
,ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’;

Let’s start with the PeriodsToDate function. The PeriodsToDate function is going to return all the sibling members at the specified level up to the current member. The outer function, the Aggregate function, calculates the value based on the aggregation type specified in the cube for the measure. In our case, the Internet Sales Amount aggregation type is Sum, so the Aggregate function could be exchanged for the Sum function.

If I wanted to calculate the Month to Date for Internet Sales, I would simply exchange the reference for the Calendar Year level of the Calendar hierarchy with the Month level, as seen here:

CREATE
MEMBER CurrentCube.[Measures].[MTD Internet Sales Amount] AS
Aggregate
(
PeriodsToDate
(
[Date].[Calendar].[Month]
,[Date].[Calendar].CurrentMember
)
,[Measures].[Internet Sales Amount]
)
,FORMAT_STRING =
Currency
,VISIBLE = 1
,DISPLAY_FOLDER = ’10 MDX Calculations’
,ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’;

9. Period over Period Growth

This is also a pretty common calculation I find and implement for my clients. It’s also pretty straight forward. In this example, we’re comparing this year’s Internet Sales Amount to the previous year’s Internet Sales Amount.

Create Member CurrentCube.[Measures].[Yearly Growth Internet Sales Amount]

As
([Date].[Calendar Year].CurrentMember,[Measures].[Internet Sales Amount])-
([Date].[Calendar Year].PrevMember,[Measures].[Internet Sales Amount]),

FORMAT_STRING = “Currency”,
VISIBLE = 1 ,  DISPLAY_FOLDER = ’10 MDX Calculations’ ,  ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’  ;

The function here to pay attention to is the PrevMember function. The PrevMember function returns the previous member at a given level based on the supplied member. For example, if the supplied member was 2011, the previous member would be 2010.

Create Member CurrentCube.[Measures].[Monthly Growth Internet Sales Amount]

As
([Date].[Month of Year].CurrentMember,[Measures].[Internet Sales Amount])-
([Date].[Month of Year].PrevMember,[Measures].[Internet Sales Amount]),

FORMAT_STRING = “Currency”,
VISIBLE = 1 , DISPLAY_FOLDER = ’10 MDX Calculations’ , ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’ ;

Similar to our last calculation, if we wish to calculate the growth from month to month, we should reference the Month attribute of our Date dimension.

8. Profit Margin Calculation

Another popular calculation for obvious reasons is the calculation for profit margin. Basically all you need to understand for this calculation is the logic of a Case statement. We use the Case statement to check for a zero value in the denominator.

CREATE
MEMBER CurrentCube.[Measures].[Internet Profit Margin] AS
CASE
WHEN
IsEmpty([Measures].[Internet Sales Amount])
THEN NULL
ELSE
([Measures].[Internet Sales Amount]

[Measures].[Internet Total Product Cost])
/
[Measures].[Internet Sales Amount]
END
,FORMAT_STRING = “Percent”
,VISIBLE = 1
,DISPLAY_FOLDER = ’10 MDX Calculations’
,ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’;

7. Percent of Total

More often then not, my clients want to be able to calculate the percent of a total amount for a hierarchy. In this example, I’m once again leveraging the Case statement.

CREATE
MEMBER CurrentCube.[Measures].[Percent of Internet Product Sales] AS
CASE
WHEN
IsEmpty([Measures].[Internet Sales Amount])
THEN NULL
ELSE
(
[Product].[Product Categories]
,[Measures].[Internet Sales Amount]
)
/
(
[Product].[Product Categories].[All]
,[Measures].[Internet Sales Amount]
)
END
,FORMAT_STRING = “Percent”
,VISIBLE = 1
,DISPLAY_FOLDER = ’10 MDX Calculations’
,ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’;

To adapt this calculation to your cube, just replace [Product].[Product Categories] with your dimension and hierarchy.

6. Count Leaf Members of a Hierarchy

I’ve seen this kind of calculation used a lot in combinations with other measures and/or calculation. This calculation counts the Leaves, which are products, of the Product Categories hierarchy.

CREATE
MEMBER CurrentCube.[Measures].[Product Count] AS
Count
(
Descendants
(
[Product].[Product Categories].CurrentMember,
,LEAVES
)
)
,FORMAT_STRING = “#,##0.00;-#,##0.00”
,VISIBLE = 1
,DISPLAY_FOLDER = ’10 MDX Calculations’
,ASSOCIATED_MEASURE_GROUP = ‘Internet Sales’;

There’s not much to this calculation. First we use the Descendants function with the Leaves function as the third argument to get all the Leaf members of the Product Categories hierarchy. Then we use the Count function to count those members.

While these calculations are not extremely complex, MDX can be a bit overwhelming. A very helpful tool that I’ve found useful, even as someone experienced with MDX, is the MDX Calculation Builder included with BI xPress. Adding calculations, like those seen above, is very easy and only take a few click with the wizard.

To open the MDX Calculation Builder, click the icon seen on the Calculations tab of your Cube Designer in BIDS.

10-30-2011 10-22-07 PM

After you click the Calculation Builder icon, the Calculation Builder wizard will open. The first thing you’ll need to do is select a calculation. At these step, we can also choose to create a named set, which the wizard will help us do.

10-30-2011 10-29-37 PM

After selecting the calculation, you’ll be taken through a couple more steps, such as selecting the Measure you wish to perform the calculation with or specifying which attribute to use as the Year attribute for the calculation.

You’ll also have the opportunity to apply conditional formatting to your measure values, which is a nice touch the end users like to see.

10-30-2011 10-46-34 PM

Finally, we’ll give our measure a name, select the measure group we’d like the measure to be a part of, select the formatting, and preview the calculation that was written by the tool.

Then we’ll click Finish. The calculation has been added to our calculation script within our cube.

10-30-2011 10-50-58 PM

BI xPress has made adding calculations to your cube simply and fast and I take advantage of the tool anytime I’m implementing calculations into my client’s cube. Head to PragmaticWorks.com to check out BI xPress and to check out some of the great training videos hosted there covering the in’s and out’s of MDX.

Stay tuned for my next blog post where we will cover the remaining 5 MDX calculations to add to your cube!

Documenting Your SSRS Reports and Data Sources

If the organizations you’ve worked with are anything like the ones I’ve had the pleasure of working with, then they probably had or have thousands of SSRS reports spread out all over the place. And back around the time a majority of companies were gearing up to migrate to SQL 2008 from 2005, getting a grasp on the number and complexity of these reports was quite a challenge.

This is just the scenario where BI Documenter really shines. Besides being able to document your SQL databases, SSIS packages, and SSAS cube, BI Documenter can also document your SSRS reports and shared data sources, all the way down to the XML behind the scenes in case your reports are lost.

Let’s walk through creating the documentation we need to understand how many and how complex the reports are that are being considered for migration. For the sake of this example, just pretend my SSRS 2008 reports are actually 2005 ;).

Open BI Documenter, click Add a New Solution. Give the solution a name and description and then click Create Solution.

1

After clicking Next, click Add Reporting Services Server on the next screen.

2

The reports you wish to document can exist either on the SSRS server or on a SharePoint Server. In my case, my reports are on my local SSRS instance so I’ll select Native for my Server Mode. Key in the server and any credentials. Click the hyperlink at the bottom of the screen to verify BI Documenter can reach the SSRS server. Click Next.

3

Give a name and description of the Report Server and click Finish.

4

Now that we’ve add the Report Server to our documentation solution, we need to create a snapshot. On the Filter tab, check the boxes next to Reports and RS Data Sources.

5

Click Next a couple times. All we need to do now is finalize the documentation. You can output the documentation in HTML format or in a .CHM file. Select the output destination.

7

On the last tab, Reporting Objects, make sure the box is checked to document the Report Definition. This is a great tool if you reports are lost. With a little work, the documentation of the Report Definition can be used to recreate the reports. Click Next. Once the process to create the documentation is finished, click Finish.

8

Now we have a precise and detailed document outlining all the reports living on our SSRS server. I can view the .rdl’s for my reports, the parameters, data sources, and data sets. I can even view the data set queries.

910

The great thing about this feature of BI Documenter is that I can easily document all my SSRS reports as long as I know the Report Server. Head here to download the free trial.

Dustin Ryan is a Data Specialist interested in Azure, SQL Server, and Power BI.

%d bloggers like this: