SSIS vs Talend

October 22, 2013

Talend vs. SSIS: A Simple Performance Comparison

As a database developer, there is often some debate about the best ETL tools on the marketplace.  This article compares SSIS to Talend Open Studio with a simple comparison testing overall ability of both tools to load 1 delimited flat file to another delimited flat file on the same server.


Converting Rows to Column with Segmentation

February 23, 2012

First of all, my apologies for the long time between posts.  My job (that pays me money) has kept me fairly busy over the past few months and the few that do follow this blog have been asking when I am going to do my next post… Well here it is.

Recently I found myself with the need to convert rows in a table into a single column.   My limitation was that I was only able to pull in 200 rows values per column and those values had to be comma delimited.  Meaning if I had 3118 rows in my source table and I needed to convert those 3118 rows into a single column, but could only have a maximum 200 row values in my column, my output would be 16 rows (3118/200 = 15.59 rounded up to 16).  My first inclination was to pivot the data; but that only allows me to create columns in excess of the values I want to pass into those columns.

Read the rest of this entry »

Shade Under The Cloud

July 20, 2011

More and more companies and consumers are increasingly using use data-intensive applications driven by cloud technology that may have been previously unavailable due to overall cost and complexity.  Now, even mom and pop shops can store data, deploy complex applications using cloud technology.  They are finding that this is often the best way to provide optimal service, reduce failure points, ensure data integrity, reduce operating costs, and create a more easily managed environment.

Read the rest of this entry »

‘Denali’ CTP3 Release

July 18, 2011

On Monday, Microsoft announced the third community technology preview (CTP3) release of its next-generation SQL Server database management system, code-named “Denali”.

I am excited to see some of the enhancements that come out with the first release:

Read the rest of this entry »

DataIntegrity is 1 Year Old!

September 22, 2010

Thats right!  This blog is officially 1.

Since I am a ‘data geek’ here are a few stats about our past revolution around the sun:

Total Visits: 6,341

Monthly Growth: 15.7%

Total Posts: 26

I would like to hear what topics you the readers would like to see before this time next year.

Context Data Mining

August 31, 2010

A lot of companies are placing big bets that location services similar to Facebook Places are going to significantly change marketing, advertising and social networking in the coming years.  While some are still skeptical, I think leveraging the data mined from location services (or from a number of other places for that matter) will help in context aware applications.  Context aware applications are capable of giving us relevant information based on where we are located and/or the activity that we are performing.  The CNN money article Data mining your digital footprints speaks about two really cool “context-aware” applications worth reading.

Read the rest of this entry »


June 29, 2010

Unfortunately SSIS does not have an SFTP connector which can obviously cause issues when trying to GET or PUT to/from an SFTP server.  Thankfully, Clement Prashant Baker posted an article regarding a workaround that does not require 3rd party software and does the job fairly well.  By downloading WinSCP, you can call their command line utility from an execute process task in SSIS.

Executable: C:\Program Files\WinSCP\

Arguments: -script=C:\DATA\USSCO\IMAGES\sftp.extract

Read the rest of this entry »

Currency Conversions in SSIS

May 3, 2010

Capturing Real-Time Currency Conversions in SSIS

This article shows how to convert currency using SSIS. A web service task makes an external call to a web service to capture up-to-date exchange rates. The really cool part about this article is that it shows how to configure: web service tasks, recordset destinations and a for-each loop to shred a recordset.

Daily Extracts… Monthly Reports

April 14, 2010

Have you ever had to run a daily task that feeds a report which is generated at the end of every month?  If your answer is yes, you probably know that this can be either a mundane task to do manually or a logistic nightmare in planning, execution & data contention.  In SSIS, I’ll demonstrate a simple way to accomplish this task while minimizing space requirements and maintenance.  The package will determine the day of the month and make a decision on which portions of the package to execute based on that day.

Read the rest of this entry »