Shade Under The Cloud

More and more companies and consumers are increasingly using use data-intensive applications driven by cloud technology that may have been previously unavailable due to overall cost and complexity.  Now, even mom and pop shops can store data, deploy complex applications using cloud technology.  They are finding that this is often the best way to provide optimal service, reduce failure points, ensure data integrity, reduce operating costs, and create a more easily managed environment.

‘Denali’ CTP3 Release

On Monday, Microsoft announced the third community technology preview (CTP3) release of its next-generation SQL Server database management system, code-named “Denali”.

I am excited to see some of the enhancements that come out with the first release:

DataIntegrity is 1 Year Old!

Thats right!  This blog is officially 1.

Since I am a ‘data geek’ here are a few stats about our past revolution around the sun:

Total Visits: 6,341

Monthly Growth: 15.7%

Total Posts: 26

I would like to hear what topics you the readers would like to see before this time next year.

Context Data Mining

A lot of companies are placing big bets that location services similar to Facebook Places are going to significantly change marketing, advertising and social networking in the coming years.  While some are still skeptical, I think leveraging the data mined from location services (or from a number of other places for that matter) will help in context aware applications.  Context aware applications are capable of giving us relevant information based on where we are located and/or the activity that we are performing.  The CNN money article Data mining your digital footprints speaks about two really cool “context-aware” applications worth reading.

Custom Error Handling in SSIS

It’s just good practice to know when your packages fail and act accordingly.  I use this gem all of the time when creating any kind of package.  I can copy these two Control Flow items into the event handler at the package level of any package and the rest takes care of itself.

SFTP Task in SSIS

Unfortunately SSIS does not have an SFTP connector which can obviously cause issues when trying to GET or PUT to/from an SFTP server.  Thankfully, Clement Prashant Baker posted an article regarding a workaround that does not require 3rd party software and does the job fairly well.  By downloading WinSCP, you can call their command line utility from an execute process task in SSIS.

Executable: C:\Program Files\WinSCP\WinSCP.com

Arguments: -script=C:\DATA\USSCO\IMAGES\sftp.extract

Currency Conversions in SSIS

Capturing Real-Time Currency Conversions in SSIS

This article shows how to convert currency using SSIS. A web service task makes an external call to a web service to capture up-to-date exchange rates. The really cool part about this article is that it shows how to configure: web service tasks, recordset destinations and a for-each loop to shred a recordset.

http://www.sqlservercentral.com/articles/Integration+Services+(SSIS)/69768/

Daily Extracts… Monthly Reports

Have you ever had to run a daily task that feeds a report which is generated at the end of every month?  If your answer is yes, you probably know that this can be either a mundane task to do manually or a logistic nightmare in planning, execution & data contention.  In SSIS, I’ll demonstrate a simple way to accomplish this task while minimizing space requirements and maintenance.  The package will determine the day of the month and make a decision on which portions of the package to execute based on that day.

BP(K*D) = i

Hey readers,  here is an excerpt from another blogger Jeff McQuigg

The full post can be found here.

I think the most interesting part about this post is discerning the difference between BI and DW.  It reminds me of the telescope/microscope analogy:  BI is about making reactive and more importantly proactive business decisions.  While some BI may involve analyzing historical trends, the newer the data, the more relevant it is.

Data Quality Part II

In Part I of this series, I showed how to create rules around data in a table. This post will expand on the idea of data quality and the need for data quality integration in your systems and applications. Ideally, a good data quality solution will help to reduce the amount of ETL (and associated overhead) required to incorporate data sets between applications, and will help to create data that is both efficient and valuable to your organization.