Categories
.NET Business Intelligence

adding custom SSIS transformation to visual studio toolbox fails

Just very recently I encountered an issue in deploying a custom SSIS component assembly where I cannot add a custom SSIS transformation to the Visual Studio toolbox.

It turns out to be a relative “no-brainer” error if only the clues were more straightforward. Basically after deploying the assembly I could not find my component listed in the “SSIS Data Flow Items” tab list. [more]

There are a number of articles available online on creating custom SSIS objects (control flow tasks, source, destination, data transformation etc).

Here are a few :

Extending SSIS Part 1 – Creating a Custom Data Flow Transformation Component

Developing a Custom Data Flow Component

Developing a Custom Transformation Component with Synchronous Outputs and searching say using keyword “adding custom SSIS transformation” should return more.

 

At Agilone, we’ve been developing SSIS components to complement our products and one of the issues I’ve faced recently is when I was deploying the custom SSIS components (the assembly) and it’s time to add it to the toolbox (Choose Items > SSIS data flow items tab (since the component was a data flow item)) I was unable to see the component (or any component on the assembly I just added to GAC and the PipelineComponents folders).

I tried these (in different orders and to no avail but for completeness including in the list):

1) close BIDS (Visual Studio Shell) or reset toolbox and close BIDS

2) uninstall the assembly in GAC and reinstall the assembly in GAC

3) make sure that assembly is on the correct folder (e.g. DTSPipelineComponent – here are more details on deployment and testing – Deploying and Testing Custom SSIS Components)

4) restart SSIS service

5) reopen the project and try to re-add the component in the toolbox again.

As mentioned it didn’t do the trick and took me sometime to figure it out.

It turns out that there was a problem with the assembly. I built the version referencing some SQL 2005 libraries (DLL), specific version = true and when I deployed it to that server (which only had SQL 2008 DLLs) it failed silently and simply didn’t appear on the “SSIS Data Flow Items” tab.

There are more complex ways to figure out why it doesn’t appear in the said tab but I would like to share the more straightforward one.

After the step where you clicked Choose Items (for toolbox) when the dialog window appears instead of changing to the “SSIS Data Flow Items” tab, stay on the .NET Components. And then click browse and browse for the assembly which contains your custom SSIS component

If the assembly chosen doesn’t contain an SSIS component then it will say so.

However if the assembly contains errors (e.g. missing referenced assemblies among others) then it will also show an error dialog saying so.

That’s it. I hope it helps and saves you some time (since I wasn’t able to find this information quickly – for a good “googler”)

And for the more “complex” method I believe you can you fuslogvw or similar assembly binding logger/monitoring tool to figure out any issues on assembly binding (missing references) such as the one I encountered. But I suggest doing he “browse” test first before exploring those debugging techniques.

And finally, of course as for any project check that you have the right assemblies referenced, whether you want specific version true/false and make sure that all dependencies are in place.

 

Categories
Business Intelligence

DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component “Fuzzy Lookup” (60) failed with error code 0xC0202009 while processing input “Fuzzy Lookup Input” (61)

Encountered this error in one of recent tasks involving SSIS Fuzzy Grouping and Lookup.

[Fuzzy Grouping Inner Data Flow : SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component “Fuzzy Lookup” (60) failed with error code 0xC0202009 while processing input “Fuzzy Lookup Input” (61). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

[Fuzzy Grouping Inner Data Flow : SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on component “OLE DB Source” (1) returned error code 0xC02020C4.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.

[Fuzzy Grouping [800]] Error: A Fuzzy Grouping transformation pipeline error occurred and returned error code 0x8000FFFF: “An unexpected error occurred.”.

This is the only relevant error message or code that I got and didn’t seem quite helpful. [more]

And it appears intermittently at first, until I noticed that the error occurs at a certain number of records and although not exactly repeatable, very close to that value. around 4M rows.

While troubleshooting, tried increasing available memory (released some memory locked for another application) and the error occurred when the number of input records increased.

Tested further and the behavior seemed consistent. And it seemed that the size of the input is proportional to the memory (RAM) used. Calculated estimated size of ever row I had to be at 1KB. Tried processing 10M rows and freed up at least 10GB of RAM and it worked fine.

I can try to work-around so that I will process smaller sets/batches so i push 10M in one pass but just needed to figure out what was causing it and although not very clear from the error message I think increasing free memory did the trick for me.

Sharing in case someone else runs into a similar error. Hope this helps