Microsoft SQL Server Community Samples: Integration Services

New Post: Binary code in the script is not found

kthejoker — Tue, 21 May 2013 16:03:01 GMT

Hi all,

I am trying to use the new EzScript component. My C# code looks like this:

        EzScript BuildNaturalKey = new EzScript(DataFlowTask1);
        BuildNaturalKey.ScriptLanguage = "CSharp";
        BuildNaturalKey.ReadOnlyVars = "User::NaturalKey";
        BuildNaturalKey.AttachTo(LoggingInfo);
        Console.Write("Script Input - " + BuildNaturalKey.InputCount.ToString());
        BuildNaturalKey.LinkAllInputsToOutputs();
        BuildNaturalKey.InitNewScript();
        BuildNaturalKey.PutSourceFile("main.cs", codeString);
       BuildNaturalKey.InsertOutputColumn("NaturalKey");
        BuildNaturalKey.SetOutputColumnDataTypeProperties("NaturalKey", DataType.DT_WSTR , 700, 0, 0, 0);
        BuildNaturalKey.Build();

Everything works fine, code runs, package is generated, but when I open it in BIDS I get the error "Binary code for the scriptis not found." When I open the script editor, the code I placed in main.cs is there, and when I simply close the window and hit okay, the script compiles.

The key difference, of course, is now the BinaryCode element of the script task's XML in the package has the binary code in it, when the original generated package is blank.

I thought the Build() command generated the BinaryCode? I see the thing in there with the BinaryCodeIndexer, but I have no how idea how to use this, and well, you guys aren't the best about documentation : )

New Post: Error: Invalid Hyperlink value in the data being updated.

SoumyadebG — Mon, 20 May 2013 13:03:35 GMT

I am using SSDT and running a package locally connecting to my DEV DB server.
I am trying to update SharePoint List Destination and insert into the same list from OLEBD Source which is running a SQL command selecting all the columns.
For both insert and update I am getting error "Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated."

Detailed error for Insert:
Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated.
Error: Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.PipelineProcessException: Errors detected in this component - see SSIS Errors
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component XXXXX failed with error code 0x80131500 while processing input "Component Input" (276). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

Detailed error for Update:
Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated. (SP ID=7868)
Error: Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.PipelineProcessException: Errors detected in this component - see SSIS Errors
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component XXXX failed with error code 0x80131500 while processing input "Component Input" (676). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

I have tried to process a single row by changing the SQL command in the Source but it is throwing the same error.
I have analyzed the data and made sure that 1 row does not contain any invalid hyperlink in it.

Please help.

Thanks
Soumyadeb

Created Unassigned: Error: Invalid Hyperlink value in the data being updated. [19735]

SoumyadebG — Mon, 20 May 2013 08:57:31 GMT

Closed Issue: Potential metadata corruption [19239]

ErikBo — Thu, 16 May 2013 11:44:57 GMT

__SharePointListAdaptersSetup - 2013-01 Beta__

gives the following error on SSDT2012 against SP2013:

Title: Editing Component

The component has detected potential metadata corruption during validation.
Error at Data Flow Task 1 [SharePoint List Source [1]]: System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.ResetConnection()
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.InitializeObject(Uri sharepointUri, NetworkCredential credential)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.GetFields(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.GetAccessibleSharePointColumns(String sharepointUrl, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.ValidateSharePointColumns()
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.Validate()
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostValidate(IDTSManagedComponentWrapper100 wrapper)

Reviewed: SharePoint List Source and Destination (May 15, 2013)

RonSteinhauser — Wed, 15 May 2013 16:30:39 GMT

Rated 1 Stars (out of 5) - Why has this not been updated to work with SQL Server Data Tools and SQL Server 2012? All of the fixes I have been able to find involve tabs that don't exist in the Tool any more. This is totally inadequate. I told my people that there was a Sharepoint solution and now I have to walk that back because it doesn't even install correctly.

New Post: Using the SSISMSBUILD extention

o0ollo0o — Fri, 10 May 2013 17:28:49 GMT

I've found my own solution and decided to share.

http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/37398f4f-530e-4561-8d1b-79cd303fdbd2/

DS19 provides a solution that works well for me

~~~~~~~
So I renounced to build it with devenv.exe command and now I'm investigating the MSBuild path: the tasks are from http://sqlsrvintegrationsrv.codeplex.com/ (main/SSISMSBuild), the build script is from http://pastebin.com/kbj8tRUz and the build command looks like below:

MSBuild SSISMSBuildDemo.msbuild /t:SSISBuild,SSISDeploy /p:SSISProj="fooProj",Configuration="Development",ProtectionLevel="EncryptSensitiveWithPassword", ProjectPassword="fooPass",SSISServer="fooServer",ProjectName="fooProjFolder"
~~~~~~~

New Post: SharePoint List Adapter with SQL Proxy

Kep — Wed, 08 May 2013 19:16:08 GMT

The error I'm getting is:

[SharePoint List Source [133]] Error: System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'NTLM'.

[SSIS.Pipeline] Error: SharePoint List Source failed validation and returned error cod 0x80131501

New Post: SharePoint List Adapter with SQL Proxy

Kep — Wed, 08 May 2013 18:52:25 GMT

I believe my situation is unique and I'm not quite sure how to approach at this time.

My SharePoint sites are housed by a separate server than the SQL server. They are also on two different domains and they aren't linked in any way at all. When I log into my computer, I log into it using the same credentials that are used for the SharePoint sites, but when I log into SQL I have a SQL user ID and password that I use and I access the SQL server from an IP Address internally.

If I run my package on my local computer it runs beautifully, however, I want to set up the package to run as a Job Server SSIS Package. The Job Server requires my SQL Log in information and is not linked in any way to the SharePoint list.

I've tried setting up a proxy SSIS Package Execution, but I'm not able to use Credentials for the SharePoint site as Credentials. It's stating an error of "Directory\User.Name" is not a valid Windows user, but it wouldn't be a Windows user.

Any direction would be appreciated...this noob is getting frustrated.

New Post: SharePoint List Adapter on SP 2013 and SQL 2012 Execution Failure

66Stang — Tue, 07 May 2013 21:42:14 GMT

When executing the SSIS package from SQL 2012 Management Studio I get the following error: "...sharepoint credential is not recognized as a valid connection manager type..."

We have packages running successfully on SQL 2008R2 using Custom Credentials (domain) and setting ProtectionLevel to EncryptAllWithPassword

SQL 2012 Setup:

SharePointListAdaptersSetup-Beta-2012-01-28.msi installed on the SQL 2012 server
VS 2012 with SQL Data Tools also installed on SQL server (for issue resolution)
Package built with VS 2012 with SharePoint List Source adapter (simple SP 2013 list)
Package executes inside VS 2012 (on the SQL Server) w/o issues
Deploy using VS to SSIS Catalog
Errors during manual execution of the Package.dtsx
Verified SharePointListAdapters.dll in C:\Program Files (x86)\Microsoft SQL Server\110\DTS\PipelineComponents
Verified SharePointUtility in GAC

On SharePoint 2013:

Everyone has Full Access rights
The service account we use for Agent job proxy has Full Access rights

Also tried:

Using Credentials of Existing Process and a scheduled Job with Proxy
Using Custom Credentials (domain) and setting ProtectionLevel to EncryptAllWithPassword

Any help will be greatly appreciated!

New Post: The HTTP request is unauthorized with client authentication scheme 'Ntlm'

ltandy — Thu, 02 May 2013 04:50:42 GMT

I'm getting the same kind of errors on a system where I don't have admin access for troubleshooting or even to view logs. I need to tell my SharePoint admins what needs to be changed but don't have a way to verify which permisions or which systems, IE or SP, need the fix.

New Post: List Rows are filtered

KarenWallace — Tue, 23 Apr 2013 19:22:59 GMT

I have list with 75 items. My data flow source has no filter on it, yet 14 items are omitted. The 14 missing rows originally all correlated to two particular values in a lookup column. Wondering if the issue was related to those values, though, I created additional values and new rows that are tied to the new values, and the rows are still missing in the output.

It's a calendar, and each entry has a start/end date, a person, and a team. The original set of missing rows are associated with two particular teams. I created new teams in the team lookup list, and created new rows in the calendar for the same people with the same start/end dates but associated with the new teams. Those new rows are also omitted by the SSIS data source. I created a new row for a different person associated with one of the new team rows. That row comes through fine. So the problem isn't the team value, and the probably isn't the person, because those people have other rows for other teams that are not omitted in the output.

Clear as mud? It doesn't make any sense to me either, but I could sure use a straw to grasp.

Commented Issue: Getting Error while trying to configuere the site URL: The HTTP request is unauthorized with client authentication scheme 'Negotiate'. [19265]

paddyprash — Wed, 17 Apr 2013 15:34:10 GMT

I'm using a SharePoint List Destination in SQL Server Integration Services. When I try to configure the site URL, via the advanced editor, I get the message see below.
I can access the Sharepoint via Browser or with the HTTP Connection Manager for HTTP Connections without any problems. Visual Studio is running as an Administrator. All Our Servers are running with NTLM authentification, i don'T know why the package is trying to establish a 'Negotiate' Connection. I'm looked in to the server with my AD-Account.

Any Ideas so solve this Issue?

Error at Data Flow Task [SharePoint List Source [1]]: System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'NTLM'. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ValidateAuthentication(HttpWebRequest request, HttpWebResponse response, WebException responseException, HttpChannelFactory factory)
at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException, ChannelBinding channelBinding)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData msgData, Int32 type)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoap.GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.ListsService_ListsSoap_GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.GetListAndView(String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointList(String listName, String viewId)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointFields(String listName, String viewId)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.GetFields(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.GetAccessibleSharePointColumns(String sharepointUrl, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.CreateExternalMetaDataColumns(IDTSOutput100 output, String sharepointUrl, String listName, String viewName, Dictionary`2 existingColumnData)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.LoadDataSourceInformation(IDTSOutput100 output, Dictionary`2 existingColumnData)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.ReinitializeMetaData()
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostReinitializeMetaData(IDTSManagedComponentWrapper100 wrapper)
Comments: ** Comment from web user: paddyprash **

I am getting this error even when we start building the data flow. Any solutions found?

New Post: Using the SSISMSBUILD extention

o0ollo0o — Thu, 11 Apr 2013 15:38:12 GMT

I'm trying to setup a build server on my local machine using the TFS Build in visual studio 2010. The build will be using a XAML file (company standard).

I'm at point where I can build a normal c# project and have my build controller, agent and definition setup correctly.

When I try to build a SSIS solution, I get the following error.
C:\Builds\10\Something.dtproj.metaproj: The project file "Something.dtproj" is not supported by MSBuild and cannot be built.

It seems that the SSISMSBUILD project in the code can resolve my issue. I'm not sure how to include it in MSBUILD and SSIS solution.

I can build the dlls with the Microsoft.SqlServer.IntegrationServices.Build.csproj file. How do I include it in MSBUILD and/or SSIS solution?

Any suggestions or advice would be appreciated.

Commented Issue: SharePoint List Adapters: Support SQL 2012 Release Version [19218]

siraj098 — Mon, 08 Apr 2013 23:20:40 GMT

Hi

We are migrating from 2008 R2 to 2012. We have made extensive use of the SharePoint List adapters to get data in an out of SharePoint lists with SSIS. Now with Data Tools I cannot get the components installed. With 2008 R2 there were tabs for Data Flow components when adding items to the toolbox. Gone in Data Tools. I have seen a few posts with people having the same problem. Has anyone got this working and if so share please. I believe the same .msi file can be used for 2012 as was used in 2008 R2. I have not attempted a reinstall as I dont want to mess up the existing 2008 packages until they get migrated.

Help!!!

Thanks
Trevor
Comments: ** Comment from web user: siraj098 **

I am having hard time using the Source List control in SSIS 2012 integration service.
Few things
No Expression Builder
No CAMLQuery . not working properly as you can only add one line.
No Field Selection
Not Working with Views.

Any help will be highly appreciated. I dont think its ready for SQL 2012

Reviewed: EzAPI - Package Generation API (SQL 2012) (abr 04, 2013)

jackh_1 — Thu, 04 Apr 2013 16:38:25 GMT

Rated 4 Stars (out of 5) - i want learn to generate pakage, thaks

New Post: Enable Forms Based Authentication when using the SharePoint List Source with Integration Project

smoro — Thu, 04 Apr 2013 15:28:30 GMT

The site I am trying to get the list data from is only setup for Form's Based Authentication and when I work with the List Control in a Visual Studio Integration Project I am getting the following:
The HTTP request was forbidden with client authentication scheme 'Ntlm'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.

Is there a way to tell the ConnectionManager not to use NTLM and use Forms Based Authentication?

Any help is appreciated.
Steve

New Post: Any way to import from multiple SharePoint sites in a loop?

stevecooperor — Thu, 21 Mar 2013 10:00:18 GMT

Hi,

I've got a number of SharePoint sites, all with the same list definition inside; I'd like to use SSIS to consolidate the data from all those sites;

I'm struggling to see if you can vary the 'SiteUrl' property of the SharePoint List Source inside the ForEach loop. I would normally expect to see an 'Expressions' property and do something like "SiteUrl : @User::sharepointUrl" but there doesn't seem to be anything like that.

Any ideas?

New Post: SharePoint List Adapter on 2012

Qucho — Tue, 19 Mar 2013 01:27:15 GMT

I had this adapter previously installed with VS 2008, and had issues making it work by simply Upgrading/Repairing the List Adapter. VS2010 would not even be able to upgrade the SSIS Package: The wizard would fail and the package would be unusable.

I had to completely remove the List Adapter from Add/Remove programs, and re-install it. Then I was able to run successfully the Upgrade Wizard on the SSIS Packages and work with them.

P.S. Love the adapter btw ;)

New Post: XML Destination

jasmindejong — Mon, 11 Mar 2013 13:53:34 GMT

The xmldest.dll works fine in sql server 2012, but I needed another encoding.
I've changed the encoding to unicode in 'xmldest.cs' in project xmlDest_SQL2012 and now I receive the following errors below:

Error: 0xC004801F at Data Flow Task, SSIS.Pipeline: The component metadata for "XML Destination, clsid {874F7595-FB5F-40FF-96AF-FBFF8250E3EF}" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning.
Error: 0xC0048021 at Data Flow Task, XML Destination [47]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
Error: 0xC0047017 at Data Flow Task, SSIS.Pipeline: XML Destination failed validation and returned error code 0xC0048021.
Error: 0xC004700C at Data Flow Task, SSIS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at Data Flow Task: There were errors during task validation.

Does anyone know what to do?

New Post: How do I import Large Lists with SharePoint List Source adapter?

Randy_D — Fri, 08 Mar 2013 23:03:07 GMT

Thanks David,
How did you do it? can you step through the steps? Did you require a loop or use CAML query to set limits?
Could it be that you have no limit restrictions on your SharePoint list? is it 2010? and could you have god mode special permission to allow you to bypaas the 5000k limit? I can get it to work for list less then 5000K no problem.