SQLSrvIntegrationSrv Forum Rss Feed

New Post: Binary code in the script is not found

kthejoker — Tue, 21 May 2013 16:03:01 GMT

Hi all,

I am trying to use the new EzScript component. My C# code looks like this:

        EzScript BuildNaturalKey = new EzScript(DataFlowTask1);
        BuildNaturalKey.ScriptLanguage = "CSharp";
        BuildNaturalKey.ReadOnlyVars = "User::NaturalKey";
        BuildNaturalKey.AttachTo(LoggingInfo);
        Console.Write("Script Input - " + BuildNaturalKey.InputCount.ToString());
        BuildNaturalKey.LinkAllInputsToOutputs();
        BuildNaturalKey.InitNewScript();
        BuildNaturalKey.PutSourceFile("main.cs", codeString);
       BuildNaturalKey.InsertOutputColumn("NaturalKey");
        BuildNaturalKey.SetOutputColumnDataTypeProperties("NaturalKey", DataType.DT_WSTR , 700, 0, 0, 0);
        BuildNaturalKey.Build();

Everything works fine, code runs, package is generated, but when I open it in BIDS I get the error "Binary code for the scriptis not found." When I open the script editor, the code I placed in main.cs is there, and when I simply close the window and hit okay, the script compiles.

The key difference, of course, is now the BinaryCode element of the script task's XML in the package has the binary code in it, when the original generated package is blank.

I thought the Build() command generated the BinaryCode? I see the thing in there with the BinaryCodeIndexer, but I have no how idea how to use this, and well, you guys aren't the best about documentation : )

New Post: Error: Invalid Hyperlink value in the data being updated.

SoumyadebG — Mon, 20 May 2013 13:03:35 GMT

I am using SSDT and running a package locally connecting to my DEV DB server.
I am trying to update SharePoint List Destination and insert into the same list from OLEBD Source which is running a SQL command selecting all the columns.
For both insert and update I am getting error "Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated."

Detailed error for Insert:
Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated.
Error: Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.PipelineProcessException: Errors detected in this component - see SSIS Errors
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component XXXXX failed with error code 0x80131500 while processing input "Component Input" (276). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

Detailed error for Update:
Error: Error on row ID="1": 0x81020020 - Invalid Hyperlink value in the data being updated. (SP ID=7868)
Error: Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.PipelineProcessException: Errors detected in this component - see SSIS Errors
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component XXXX failed with error code 0x80131500 while processing input "Component Input" (676). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

I have tried to process a single row by changing the SQL command in the Source but it is throwing the same error.
I have analyzed the data and made sure that 1 row does not contain any invalid hyperlink in it.

Please help.

Thanks
Soumyadeb

New Post: Using the SSISMSBUILD extention

o0ollo0o — Fri, 10 May 2013 17:28:49 GMT

I've found my own solution and decided to share.

http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/37398f4f-530e-4561-8d1b-79cd303fdbd2/

DS19 provides a solution that works well for me

~~~~~~~
So I renounced to build it with devenv.exe command and now I'm investigating the MSBuild path: the tasks are from http://sqlsrvintegrationsrv.codeplex.com/ (main/SSISMSBuild), the build script is from http://pastebin.com/kbj8tRUz and the build command looks like below:

MSBuild SSISMSBuildDemo.msbuild /t:SSISBuild,SSISDeploy /p:SSISProj="fooProj",Configuration="Development",ProtectionLevel="EncryptSensitiveWithPassword", ProjectPassword="fooPass",SSISServer="fooServer",ProjectName="fooProjFolder"
~~~~~~~

New Post: SharePoint List Adapter with SQL Proxy

Kep — Wed, 08 May 2013 19:16:08 GMT

The error I'm getting is:

[SharePoint List Source [133]] Error: System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'NTLM'.

[SSIS.Pipeline] Error: SharePoint List Source failed validation and returned error cod 0x80131501

New Post: SharePoint List Adapter with SQL Proxy

Kep — Wed, 08 May 2013 18:52:25 GMT

I believe my situation is unique and I'm not quite sure how to approach at this time.

My SharePoint sites are housed by a separate server than the SQL server. They are also on two different domains and they aren't linked in any way at all. When I log into my computer, I log into it using the same credentials that are used for the SharePoint sites, but when I log into SQL I have a SQL user ID and password that I use and I access the SQL server from an IP Address internally.

If I run my package on my local computer it runs beautifully, however, I want to set up the package to run as a Job Server SSIS Package. The Job Server requires my SQL Log in information and is not linked in any way to the SharePoint list.

I've tried setting up a proxy SSIS Package Execution, but I'm not able to use Credentials for the SharePoint site as Credentials. It's stating an error of "Directory\User.Name" is not a valid Windows user, but it wouldn't be a Windows user.

Any direction would be appreciated...this noob is getting frustrated.

New Post: SharePoint List Adapter on SP 2013 and SQL 2012 Execution Failure

66Stang — Tue, 07 May 2013 21:42:14 GMT

When executing the SSIS package from SQL 2012 Management Studio I get the following error: "...sharepoint credential is not recognized as a valid connection manager type..."

We have packages running successfully on SQL 2008R2 using Custom Credentials (domain) and setting ProtectionLevel to EncryptAllWithPassword

SQL 2012 Setup:

SharePointListAdaptersSetup-Beta-2012-01-28.msi installed on the SQL 2012 server
VS 2012 with SQL Data Tools also installed on SQL server (for issue resolution)
Package built with VS 2012 with SharePoint List Source adapter (simple SP 2013 list)
Package executes inside VS 2012 (on the SQL Server) w/o issues
Deploy using VS to SSIS Catalog
Errors during manual execution of the Package.dtsx
Verified SharePointListAdapters.dll in C:\Program Files (x86)\Microsoft SQL Server\110\DTS\PipelineComponents
Verified SharePointUtility in GAC

On SharePoint 2013:

Everyone has Full Access rights
The service account we use for Agent job proxy has Full Access rights

Also tried:

Using Credentials of Existing Process and a scheduled Job with Proxy
Using Custom Credentials (domain) and setting ProtectionLevel to EncryptAllWithPassword

Any help will be greatly appreciated!

New Post: The HTTP request is unauthorized with client authentication scheme 'Ntlm'

ltandy — Thu, 02 May 2013 04:50:42 GMT

I'm getting the same kind of errors on a system where I don't have admin access for troubleshooting or even to view logs. I need to tell my SharePoint admins what needs to be changed but don't have a way to verify which permisions or which systems, IE or SP, need the fix.

New Post: List Rows are filtered

KarenWallace — Tue, 23 Apr 2013 19:22:59 GMT

I have list with 75 items. My data flow source has no filter on it, yet 14 items are omitted. The 14 missing rows originally all correlated to two particular values in a lookup column. Wondering if the issue was related to those values, though, I created additional values and new rows that are tied to the new values, and the rows are still missing in the output.

It's a calendar, and each entry has a start/end date, a person, and a team. The original set of missing rows are associated with two particular teams. I created new teams in the team lookup list, and created new rows in the calendar for the same people with the same start/end dates but associated with the new teams. Those new rows are also omitted by the SSIS data source. I created a new row for a different person associated with one of the new team rows. That row comes through fine. So the problem isn't the team value, and the probably isn't the person, because those people have other rows for other teams that are not omitted in the output.

Clear as mud? It doesn't make any sense to me either, but I could sure use a straw to grasp.

New Post: Using the SSISMSBUILD extention

o0ollo0o — Thu, 11 Apr 2013 15:38:12 GMT

I'm trying to setup a build server on my local machine using the TFS Build in visual studio 2010. The build will be using a XAML file (company standard).

I'm at point where I can build a normal c# project and have my build controller, agent and definition setup correctly.

When I try to build a SSIS solution, I get the following error.
C:\Builds\10\Something.dtproj.metaproj: The project file "Something.dtproj" is not supported by MSBuild and cannot be built.

It seems that the SSISMSBUILD project in the code can resolve my issue. I'm not sure how to include it in MSBUILD and SSIS solution.

I can build the dlls with the Microsoft.SqlServer.IntegrationServices.Build.csproj file. How do I include it in MSBUILD and/or SSIS solution?

Any suggestions or advice would be appreciated.

New Post: Enable Forms Based Authentication when using the SharePoint List Source with Integration Project

smoro — Thu, 04 Apr 2013 15:28:30 GMT

The site I am trying to get the list data from is only setup for Form's Based Authentication and when I work with the List Control in a Visual Studio Integration Project I am getting the following:
The HTTP request was forbidden with client authentication scheme 'Ntlm'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.

Is there a way to tell the ConnectionManager not to use NTLM and use Forms Based Authentication?

Any help is appreciated.
Steve

New Post: Any way to import from multiple SharePoint sites in a loop?

stevecooperor — Thu, 21 Mar 2013 10:00:18 GMT

Hi,

I've got a number of SharePoint sites, all with the same list definition inside; I'd like to use SSIS to consolidate the data from all those sites;

I'm struggling to see if you can vary the 'SiteUrl' property of the SharePoint List Source inside the ForEach loop. I would normally expect to see an 'Expressions' property and do something like "SiteUrl : @User::sharepointUrl" but there doesn't seem to be anything like that.

Any ideas?

New Post: SharePoint List Adapter on 2012

Qucho — Tue, 19 Mar 2013 01:27:15 GMT

I had this adapter previously installed with VS 2008, and had issues making it work by simply Upgrading/Repairing the List Adapter. VS2010 would not even be able to upgrade the SSIS Package: The wizard would fail and the package would be unusable.

I had to completely remove the List Adapter from Add/Remove programs, and re-install it. Then I was able to run successfully the Upgrade Wizard on the SSIS Packages and work with them.

P.S. Love the adapter btw ;)

New Post: XML Destination

jasmindejong — Mon, 11 Mar 2013 13:53:34 GMT

The xmldest.dll works fine in sql server 2012, but I needed another encoding.
I've changed the encoding to unicode in 'xmldest.cs' in project xmlDest_SQL2012 and now I receive the following errors below:

Error: 0xC004801F at Data Flow Task, SSIS.Pipeline: The component metadata for "XML Destination, clsid {874F7595-FB5F-40FF-96AF-FBFF8250E3EF}" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning.
Error: 0xC0048021 at Data Flow Task, XML Destination [47]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
Error: 0xC0047017 at Data Flow Task, SSIS.Pipeline: XML Destination failed validation and returned error code 0xC0048021.
Error: 0xC004700C at Data Flow Task, SSIS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at Data Flow Task: There were errors during task validation.

Does anyone know what to do?

New Post: How do I import Large Lists with SharePoint List Source adapter?

Randy_D — Fri, 08 Mar 2013 23:03:07 GMT

Thanks David,
How did you do it? can you step through the steps? Did you require a loop or use CAML query to set limits?
Could it be that you have no limit restrictions on your SharePoint list? is it 2010? and could you have god mode special permission to allow you to bypaas the 5000k limit? I can get it to work for list less then 5000K no problem.

New Post: The HTTP request is unauthorized with client authentication scheme 'Ntlm'

adsz889 — Fri, 08 Mar 2013 03:46:01 GMT

I am using the SharePoint source list adapter and when I try to connect to any list in SharePoint 2007, I got 401 error. This happens in both designer and SQL Agent, and I am sure that my account and the SQL agent account has permission to the SharePoint list.
It works with all other Web Applications within the farm, but fail for all sites within this particular webapp that I need data from. All webapp in the farm are using Windows Integrated Auth with Kerberos.

Error at Data Flow Task [SharePoint List Source [1]]: System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'Negotiate,NTLM'. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ValidateAuthentication(HttpWebRequest request, HttpWebResponse response, WebException responseException, HttpChannelFactory factory)
at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException, ChannelBinding channelBinding)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoap.GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.ListsService_ListsSoap_GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.GetListAndView(String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointList(String listName, String viewId)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointFields(String listName, String viewId)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.GetFields(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.GetAccessibleSharePointColumns(String sharepointUrl, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.ValidateSharePointColumns()
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.Validate()
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostValidate(IDTSManagedComponentWrapper100 wrapper)
Error at Data Flow Task [SSIS.Pipeline]: component "SharePoint List Source" (1) failed validation and returned error code 0x80131501.

New Post: How do I import Large Lists with SharePoint List Source adapter?

DavidBridge — Thu, 07 Mar 2013 14:53:30 GMT

I don't know the answer to your question but the lists I have been bringing out atr 21 columns and 150,000 rows with a batch of 1,000 and the adapter coped fine.

It brought 355K rows in 34 minutes

Dave

New Post: System.ServiceModel.FaultException: Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException' was thrown.

kmjones77 — Wed, 27 Feb 2013 09:58:28 GMT

When querying a SharePoint list using the SharePoint List Adapter in SSIS, I get the following message:

Error: 0xC0047062 at SharepointDocument Population Staging, SharePoint List Source [4444]: Microsoft.Samples.SqlServer.SSIS.SharePointUtility.SharePointUnhandledException: Unspecified SharePoint Error. A possible reason might be you are trying to retrieve too many items at a time (Batch size) ---> System.ServiceModel.FaultException: Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException' was thrown.

It would be nice to be able to see the InnerException detail here, is that possible?

Thanks,

Ken

New Post: SharePoint List Source not returning records

bioforge — Wed, 27 Feb 2013 08:06:45 GMT

I have the same problem too, no error but no data is read. I have put the farm admin user and password in the connection manager but I am running the package from BIDS installed on the sql server while logged in with another account. There aren't any item-level premissions. This message shows in the progress tab.

[SharePoint List Source] Information: Loaded 0 records from list 'listname' at 'url'. Elapsed time is 190ms

any suggestions ?

New Post: Anyone figure this out with Sharepoint Online/Office 365?

Skutr — Wed, 27 Feb 2013 00:17:42 GMT

I'm stuck here as well. I am sending text files to a doc library in 365 via email and need to pull the files down into SSIS for ETL processing..... keep getting the FORBIDDEN error 403.
Has anyone made any progress on this front?

New Post: Error in SharePoint List Destination: System.InvalidOperationException: Duplicate attribute

chasejace — Tue, 26 Feb 2013 19:00:26 GMT

Hi All,

I am currently utilizing the SharePoint List Destination component in my SSIS package to transfer metadata to a SharePoint list. However, I am currently getting this error when I run my package:

[Update Sharepoint Metadata [4281]]
Error: System.InvalidOperationException: Duplicate attribute.
at System.Xml.Linq.XElement.AddAttributeSkipNotify(XAttribute a)
at System.Xml.Linq.XContainer.AddContentSkipNotify(Object content)
at System.Xml.Linq.XContainer.AddContentSkipNotify(Object content)
at System.Xml.Linq.XContainer.Add(Object content)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter._Closure$__7._Lambda$__63(XElement row)
at System.Linq.Enumerable.WhereSelectEnumerableIterator

2.MoveNext()    

at System.Linq.Enumerable.<UnionIterator>d__81

1.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator

2.MoveNext()    

at System.Xml.Linq.XContainer.AddContentSkipNotify(Object content)    

at System.Xml.Linq.XContainer.Add(Object content)    

at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.ExecuteSharePointUpdateBatch(String listName, String viewId, XElement batchXml, Int16 batchSize)    

at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.UpdateListItems(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName, IEnumerable

1 fieldValueList, Int16 batchSize)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper90 wrapper, Int32 inputID, IDTSBuffer90 pDTSBuffer, IntPtr bufferWirePacket)

Appreciate your kind assistance on this matter. Thanks a lot! :)