SQLSrvIntegrationSrv Wiki & Documentation Rss Feed

New Comment on "SharePoint List Adapters"

BobSanders — Wed, 13 Feb 2013 13:17:30 GMT

Is there any sample code of dynamically creating a CAML query using an expression - for example to pass in a "last-run" date to use for incremental loads?

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Wed, 23 Jan 2013 17:16:41 GMT

SharePoint List Adapter Changelog

1/23/2013
1) Optimized memory consumption during runtime (patch from dbennet84)
2) Fix for missing header property for ADFS (snippets from TechyGuy)
3) Fix for RemoveFiles to make it more robust (snippets from cpassuel)
4) Fix for race condition in columndata with multiple data flows (kmartin)
5) Fix for working in Claims environment (chadc)
Thank you to all of the contributors, who provided all of the fixes provided in this update! A Beta is available for testing by the end users.

10/17/2011
1) Allowing connection manager to be disabled for compat scenarios reported using sql 2005
2) Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
3) Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
4) Added a property to retrieve the Hidden columns of a SharePoint List
5) Found an issue with SharePointUtility deployment where the latest was not being deployed (Possible reason some did not see features posted since June)
6) Added support for "DecodeLookupColumns". Toggle it false and refresh to get the raw Lookup/Choice columns, toggle it true to have those columns removed and replaced with two new columns - one for the ID(s) and one for the Value(s).
For Custom Choice columns, there will just be a value field (editable) which will be a newline separated list of string values (no ;# characters)
For Lookup columns, there will be an ID field (editable) with a newlien separated list of values and a Value field (readonly). Using other datasources, you can cross reference using CAML or whatever other data points to match IDs.
Note, changing the editable field means that you must enter the number or value in it using a newline separated string for it to be processed on the update. You cannot have both the Raw field and the Translated field in a dataflow.
THANK you to user leiyu who submitted a changeset with this functionality, to help initiate this new feature.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapters

kevinidzi — Wed, 23 Jan 2013 13:55:23 GMT

Common Questions / Answers

MSDN Documentation: http://msdn.microsoft.com/en-us/library/hh368261.aspx
MVP Article Sample (Ivan Sanders): http://msmvps.com/blogs/ivansanders/archive/2011/07/24/ssis-sharepoint-list-adapters.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: After upgrading SQL 2008 -> 2012, I'm not seeing the adapters - How to fix?

Upgrade the package from SQL 2008 to SQL 2012. I did this by opening the project in Data Tools and following the Upgrade wizard
You will get a nasty message about the SharePoint list component
If you look in C:\Program Files\Microsoft SQL Server\100\DTS\PipelineComponents you will see SharePointListAdapters.dll
Re-install the SharePointList adapterc
The same dll will now also exist in C:\Program Files\Microsoft SQL Server\110\DTS\PipeLineComponents
Open up Data Tools and the package that has been upgraded
You will notice an error condition on the data flow component that contains the task for the SharePoint source on the control fow
Switch to this data flow component
Right click and select SSIS Toolbox
Important - click on the Toolbox panel and choose Refresh Toolbox
Hopefully you should see both the source and destination components in the common section of the toobox
Unfortunately you have to redo the settings for the SharePoint component. Its a bit of a pain but a bit of copy and paste from the old SQL 2008 solution (which you should have backed up before the migration) gets the package working in a couple of minutes.

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

Q: OR... The list adapters work in Design mode, but do not work at runtime (Unauthorized error).

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Q: I am not seeing any of the new components after installation?

If you had VS opened during installation, then be sure to close it and reopen it to see the sharepoint components.

Pulling Data from SharePoint

Q: I'm getting this error: System.Net.WebException: The remote server returned an error: (417) Expectation failed.

Check your proxy settings in IE on the machine executing the DTS Package, as one user indicated Squid proxy settings caused this issue for them.

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: Home

MattCarroll — Wed, 18 Apr 2012 23:24:23 GMT

Project Description
Microsoft SQL Server Integration Services (SSIS) makes it possible to build high performance data integration solutions, including the extraction, transformation, and loading (ETL) of data for data warehousing and as well as Data Transformation Services (DTS) backward compatible support.

This project contains Integration Services samples created by the product team. If you have questions or suggestions about any of these samples or Integration Services, please let us know by posting in the SSIS forum.

For recent Integration Services downloads, samples, articles, videos and more from Microsoft and the community, click here.

Release Information

Each sample has its own release on the site. Each release contains an MSI that make it easy to install and get working. In most cases, only the binary is provided by the MSI. To view the source code, click the Source Code tab above.

Unless otherwise noted, all samples are built for SQL Server 2008.

XML Destination Sample

This sample includes source and binary for a simple XML Destination pipeline component. Use this sample to learn more about how to:

Create custom data flow destination components for use with SSIS
Build component user interfaces
Support multiple inputs on a single component

Written by David Noor

(Update to support SQL Server 2012 added)

Regular Expression Flat File Source Sample

This sample includes source and binary for a regular expression based flat file parsing source. Use this sample to learn more about how to:

Create custom data flow source components for use with SSIS
Support multiple outputs from a single component
Define output columns
Validate metadata

Written by Silviu Guea

(Update to support SQL Server 2012 added)

Delimited File Reader Source Sample

This sample includes the binaries for a source component capable of parsing delimited flat files, including files with rows that are missing column fields. Use this sample to learn more about how to:

Create custom data flow source components for use with SSIS
Use Visual Studio Test Edition-based unit tests to perfom automated unit testing on data flow components
Respond to and perform custom UI interaction when a component is dragged to the design surface
Redirect error rows to an error output

Written by Bob Bojanic

(Update to support SQL Server 2012 added)

Package Generation Sample

This sample demonstrates package generation and execution using the Integration Services object model. The sample can be used to transfer data between a pair of source and destination components at the command line. The sample supports three types of source and destination components: SQL Server, Excel and flat file. Use this sample to learn more about how to:

Create packages using the SSIS API
Validate packages using the SSIS API
Execute packages using the SSIS API
Add type conversions using the SSIS API

Written by Jia Li

Hello World Task Sample

This sample shows how to create a simple task that extends the SSIS base UI classes to provide a custom UI with the same look and feel as the SSIS tasks that ship with SQL Server 2008. It also shows:

Custom UI: The task extends the DTSBaseTaskUI class to have a simple UI with the same look and feel as other SSIS tasks.

Written by Matt Masson

(Update to support SQL Server 2012 added)

SharePoint List Source and Destination Sample

MSDN Tutorial
This sample demonstrates how to get data into and out of SharePoint lists by using custom source and destination adapters written in C# 3.0. Use this sample to learn more about:

Extensive Validation: Validation for this component actively goes against the SharePoint Site to verify the properties are valid.
Values from Expressions: This component supports external variables, which can be expressions, and can be attached to the source component to customize the query. Similar to the CommandText for the other Sql Components
Linq: The Component has been written using Linq with .net 3.5 and shows how elements such as the metadata and columns can be combined to create a readable usage in a Linq format.
Custom properties: The component keeps its configuration in custom properties on itself, inputs, and input columns.
FAQ: http://sqlsrvintegrationsrv.codeplex.com/wikipage?title=SharePoint%20List%20Adapters&referringTitle=Documentation

Written by Kevin Idzi

WMI Source Component Samples

This sample demonstrates how to use WMI (Windows Management Instrumentation) as a data source. Using WMI, you can retrieve and process all kinds of data about your system's performance using SSIS data flows. Use this sample to learn more about:

Data type mapping: Correctly mapping data between custom types and SSIS types
Managing external metadata
Interacting with WMI

Written by Alexander Vikhorev

(Update to support SQL Server 2012 added)

Spatial Data Flow Components

This sample contains three components showing how to use spatial data within synchronous and asynchronous data flow transforms:

Spatial Grid: replicating rows with SqlGeometry data by cutting geometry object in pieces on a given grid
Spatial Union: aggregation of spatial data grouped by a regular column. The sample is simplified by requiring a sorted group by column.
Vector Transformations: applying series of transformations (translations, rotations and scaling), defined using simple expressions, on geometry objects contained in a SqlGeometry column.

Written by Bob Bojanic

MERGE Destination

This sample contains a custom component showing how to combine ADO.NET's new table-valued parameter support, the new SQL MERGE statement, and SSIS together to create a powerful and fast component for performing MERGE operations against SQL Server. Use this sample to learn more about:

Writing custom data flow components
Error redirection
Using new ADO.NET TVP support

Written by Sourav Mukherjee and Terri Chen

(Update to support SQL Server 2012 added)

BizTalk to Integration Services Samples

BizTalk Server is designed to help users easily integrate various applications and systems to talk to each other. These samples demonstrate how to integrate SQL Server Integration Services 2005 with BizTalk 2006. They show you how to:

Invoke an SSIS package from a BizTalk orchestration using an expression shape
Invoke an SSIS package from a BizTalk pipeline using a custom pipeline component
Pass information from BizTalk to SSIS

It is recommended that you read the BizTalkIntegationSamplesReadme.rtf document below before installing the sample.

Written by Cho Yeung

EzAPI - Package Generation API

This sample provides some functionality to easily create SSIS packages programmatically and dynamically alter their objects (tasks, components, changing metadata, connection strings, etc). This framework supports:

Creation SSIS packages of any complexity including both SSIS runtime and pipeline (tasks, containers and components)
BIDS like behavior (automatic column mapping in destinations, automatic metadata refresh, default values of properties, etc)

Written by Evgeny Koblov

(Update to support SQL Server 2012 added)

Updated Wiki: Home

dnoor — Fri, 09 Dec 2011 00:34:42 GMT

Release Information

XML Destination Sample

This sample includes source and binary for a simple XML Destination pipeline component. Use this sample to learn more about how to:

Create custom data flow destination components for use with SSIS
Build component user interfaces
Support multiple inputs on a single component

Written by David Noor

Regular Expression Flat File Source Sample

This sample includes source and binary for a regular expression based flat file parsing source. Use this sample to learn more about how to:

Create custom data flow source components for use with SSIS
Support multiple outputs from a single component
Define output columns
Validate metadata

Written by Silviu Guea

Delimited File Reader Source Sample

This sample includes the binaries for a source component capable of parsing delimited flat files, including files with rows that are missing column fields. Use this sample to learn more about how to:

Create custom data flow source components for use with SSIS
Use Visual Studio Test Edition-based unit tests to perfom automated unit testing on data flow components
Respond to and perform custom UI interaction when a component is dragged to the design surface
Redirect error rows to an error output

Written by Bob Bojanic

Package Generation Sample

Create packages using the SSIS API
Validate packages using the SSIS API
Execute packages using the SSIS API
Add type conversions using the SSIS API

Written by Jia Li

Hello World Task Sample

This sample shows how to create a simple task that extends the SSIS base UI classes to provide a custom UI with the same look and feel as the SSIS tasks that ship with SQL Server 2008. It also shows:

Custom UI: The task extends the DTSBaseTaskUI class to have a simple UI with the same look and feel as other SSIS tasks.

Written by Matt Masson

SharePoint List Source and Destination Sample

MSDN Tutorial
This sample demonstrates how to get data into and out of SharePoint lists by using custom source and destination adapters written in C# 3.0. Use this sample to learn more about:

Extensive Validation: Validation for this component actively goes against the SharePoint Site to verify the properties are valid.
Values from Expressions: This component supports external variables, which can be expressions, and can be attached to the source component to customize the query. Similar to the CommandText for the other Sql Components
Linq: The Component has been written using Linq with .net 3.5 and shows how elements such as the metadata and columns can be combined to create a readable usage in a Linq format.
Custom properties: The component keeps its configuration in custom properties on itself, inputs, and input columns.
FAQ: http://sqlsrvintegrationsrv.codeplex.com/wikipage?title=SharePoint%20List%20Adapters&referringTitle=Documentation

Written by Kevin Idzi

WMI Source Component Samples

Data type mapping: Correctly mapping data between custom types and SSIS types
Managing external metadata
Interacting with WMI

Written by Alexander Vikhorev

Spatial Data Flow Components

This sample contains three components showing how to use spatial data within synchronous and asynchronous data flow transforms:

Spatial Grid: replicating rows with SqlGeometry data by cutting geometry object in pieces on a given grid
Spatial Union: aggregation of spatial data grouped by a regular column. The sample is simplified by requiring a sorted group by column.
Vector Transformations: applying series of transformations (translations, rotations and scaling), defined using simple expressions, on geometry objects contained in a SqlGeometry column.

Written by Bob Bojanic

MERGE Destination

Writing custom data flow components
Error redirection
Using new ADO.NET TVP support

Written by Sourav Mukherjee and Terri Chen

BizTalk to Integration Services Samples

Invoke an SSIS package from a BizTalk orchestration using an expression shape
Invoke an SSIS package from a BizTalk pipeline using a custom pipeline component
Pass information from BizTalk to SSIS

It is recommended that you read the BizTalkIntegationSamplesReadme.rtf document below before installing the sample.

Written by Cho Yeung

EzAPI - Package Generation API

Creation SSIS packages of any complexity including both SSIS runtime and pipeline (tasks, containers and components)
BIDS like behavior (automatic column mapping in destinations, automatic metadata refresh, default values of properties, etc)

Written by Evgeny Koblov

Updated Wiki: SharePoint List Adapters

kevinidzi — Fri, 25 Nov 2011 05:01:33 GMT

Common Questions / Answers

MSDN Documentation: http://msdn.microsoft.com/en-us/library/hh368261.aspx
MVP Article Sample (Ivan Sanders): http://msmvps.com/blogs/ivansanders/archive/2011/07/24/ssis-sharepoint-list-adapters.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

Q: OR... The list adapters work in Design mode, but do not work at runtime (Unauthorized error).

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Q: I am not seeing any of the new components after installation?

If you had VS opened during installation, then be sure to close it and reopen it to see the sharepoint components.

Pulling Data from SharePoint

Q: I'm getting this error: System.Net.WebException: The remote server returned an error: (417) Expectation failed.

Check your proxy settings in IE on the machine executing the DTS Package, as one user indicated Squid proxy settings caused this issue for them.

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: SharePoint List Adapters

kevinidzi — Fri, 25 Nov 2011 04:54:20 GMT

Common Questions / Answers

MSDN Documentation: http://msdn.microsoft.com/en-us/library/hh368261.aspx
MVP Article Sample (Ivan Sanders): http://msmvps.com/blogs/ivansanders/archive/2011/07/24/ssis-sharepoint-list-adapters.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Q: I am not seeing any of the new components after installation?

If you had VS opened during installation, then be sure to close it and reopen it to see the sharepoint components.

Pulling Data from SharePoint

Q: I'm getting this error: System.Net.WebException: The remote server returned an error: (417) Expectation failed.

Check your proxy settings in IE on the machine executing the DTS Package, as one user indicated Squid proxy settings caused this issue for them.

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Mon, 17 Oct 2011 16:28:30 GMT

SharePoint List Adapter Changelog

10/17/2011
1) Allowing connection manager to be disabled for compat scenarios reported using sql 2005
2) Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
3) Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
4) Added a property to retrieve the Hidden columns of a SharePoint List
5) Found an issue with SharePointUtility deployment where the latest was not being deployed (Possible reason some did not see features posted since June)
6) Added support for "DecodeLookupColumns". Toggle it false and refresh to get the raw Lookup/Choice columns, toggle it true to have those columns removed and replaced with two new columns - one for the ID(s) and one for the Value(s).
For Custom Choice columns, there will just be a value field (editable) which will be a newline separated list of string values (no ;# characters)
For Lookup columns, there will be an ID field (editable) with a newlien separated list of values and a Value field (readonly). Using other datasources, you can cross reference using CAML or whatever other data points to match IDs.
Note, changing the editable field means that you must enter the number or value in it using a newline separated string for it to be processed on the update. You cannot have both the Raw field and the Translated field in a dataflow.
THANK you to user leiyu who submitted a changeset with this functionality, to help initiate this new feature.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: Documentation

kevinidzi — Fri, 14 Oct 2011 07:16:10 GMT

FAQ

SharePoint List Adapters
- SharePoint List Adapters
- SharePoint List Adapter Release Notes

Updated Wiki: Documentation

kevinidzi — Fri, 14 Oct 2011 07:15:59 GMT

FAQ

SharePoint List Adapters
- SharePoint List Adapters FAQ
- SharePoint List Adapter Release Notes

Updated Wiki: SharePoint List Adapters

kevinidzi — Fri, 14 Oct 2011 07:15:24 GMT

Common Questions / Answers

MSDN Documentaiton: http://msdn.microsoft.com/en-us/library/hh368261.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Q: I am not seeing any of the new components after installation?

If you had VS opened during installation, then be sure to close it and reopen it to see the sharepoint components.

Pulling Data from SharePoint

Q: I'm getting this error: System.Net.WebException: The remote server returned an error: (417) Expectation failed.

Check your proxy settings in IE on the machine executing the DTS Package, as one user indicated Squid proxy settings caused this issue for them.

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Wed, 12 Oct 2011 22:20:33 GMT

SharePoint List Adapter Changelog

10/6/2011 BETA Release
1) Allowing connection manager to be disabled for compat scenarios reported using sql 2005
2) Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
3) Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
4) Added a property to retrieve the Hidden columns of a SharePoint List
5) Found an issue with SharePointUtility deployment where the latest was not being deployed (Possible reason some did not see features posted since June)
6) Added support for "DecodeLookupColumns". Toggle it false and refresh to get the raw Lookup/Choice columns, toggle it true to have those columns removed and replaced with two new columns - one for the ID(s) and one for the Value(s).
For Custom Choice columns, there will just be a value field (editable) which will be a newline separated list of string values (no ;# characters)
For Lookup columns, there will be an ID field (editable) with a newlien separated list of values and a Value field (readonly). Using other datasources, you can cross reference using CAML or whatever other data points to match IDs.
Note, changing the editable field means that you must enter the number or value in it using a newline separated string for it to be processed on the update. You cannot have both the Raw field and the Translated field in a dataflow.
THANK you to user leiyu who submitted a changeset with this functionality, to help initiate this new feature.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Wed, 12 Oct 2011 22:19:53 GMT

SharePoint List Adapter Changelog

10/6/2011
1) Beta versions available for testing two scenarios:
- Allowing connection manager to be disabled for compat scenarios reported using sql 2005
- Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
- Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
- Added a property to retrieve the Hidden columns of a SharePoint List
- Found an issue with SharePointUtility deployment where the latest was not being deployed (Possible reason some did not see features posted since June)
- Added support for "DecodeLookupColumns". Toggle it false and refresh to get the raw Lookup/Choice columns, toggle it true to have those columns removed and replaced with two new columns - one for the ID(s) and one for the Value(s).
For Custom Choice columns, there will just be a value field (editable) which will be a newline separated list of string values (no ;# characters)
For Lookup columns, there will be an ID field (editable) with a newlien separated list of values and a Value field (readonly). Using other datasources, you can cross reference using CAML or whatever other data points to match IDs.
Note, changing the editable field means that you must enter the number or value in it using a newline separated string for it to be processed on the update. You cannot have both the Raw field and the Translated field in a dataflow.
THANK you to user leiyu who submitted a changeset with this functionality, to help initiate this new feature.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Wed, 12 Oct 2011 20:32:55 GMT

SharePoint List Adapter Changelog

10/6/2011
1) Beta versions available for testing two scenarios:
- Allowing connection manager to be disabled for compat scenarios reported using sql 2005
- Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
- Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
- Added a property to retrieve the Hidden columns of a SharePoint List
- Found an issue with SharePointUtility deployment where the latest was not being deployed (Possible reason some did not see features posted since June)
- Added support for "DecodeLookupColumns". Toggle it false and refresh to get the raw Lookup/Choice columns, toggle it true to have those columns removed and replaced with two new columns - one for the ID(s) and one for the Value(s).
For Custom Choice columns, there will just be a value field (editable) which will be a newline separated list of string values (no ;# characters)
For Lookup columns, there will be an ID field (editable) with a newlien separated list of values and a Value field (readonly). Using other datasources, you can cross reference using CAML or whatever other data points to match IDs.
Note, changing the editable field means that you must enter the number or value in it using a newline separated string for it to be processed on the update. You cannot have both the Raw field and the Translated field in a dataflow.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapters

kevinidzi — Fri, 07 Oct 2011 15:43:17 GMT

Common Questions / Answers

MSDN Documentaiton: http://msdn.microsoft.com/en-us/library/hh368261.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Q: I am not seeing any of the new components after installation?

If you had VS opened during installation, then be sure to close it and reopen it to see the sharepoint components.

Pulling Data from SharePoint

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Thu, 06 Oct 2011 19:29:21 GMT

SharePoint List Adapter Changelog

10/6/2011
1) Beta versions available for testing two scenarios:
- Allowing connection manager to be disabled for compat scenarios reported using sql 2005
- Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.
- Fixed a connection close issue which may have occured when using Sharepoint List Views to filter the results of a list
- Added a property to retrieve the Hidden columns of a SharePoint List

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Thu, 06 Oct 2011 16:46:10 GMT

SharePoint List Adapter Changelog

10/6/2011
1) Beta versions available for testing two scenarios:
- Allowing connection manager to be disabled for compat scenarios reported using sql 2005
- Fixing duplicated column names from SharePoint making them unique to reduce friction for setting up adapter.

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapters

kevinidzi — Wed, 14 Sep 2011 07:52:24 GMT

Common Questions / Answers

MSDN Documentaiton: http://msdn.microsoft.com/en-us/library/hh368261.aspx

Setup and Execution

Q: What version of Windows Installer is needed to install the adapters?

You need Windows Installer 4.5 - http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8483

Q: I'm getting PerformUpgrade errors and with the new adapters, my components do not load, help!

A one-time publickey change was made which requires you to replace: 933a2c7edf82ac1f with f4b3011e1ece9d47 in your DTSX files.

Q: The SharePoint Adapters do not show up in my toolbox after installation

This can happen when the components are installed near the SQL engine, but not on the drive with the actual SSIS components. Here is the workaround:
1. Open Business Intelligence Development Studio, open a package, and then click Choose Toolbox Items on the Tools menu.
2. In the Choose Toolbox Items dialog box, click the SSIS Data Flow Items tab, and then check SharePoint Destination and SharePoint Source and click OK.
3. Or, using the above steps, look at the other components in the toolbox to get an idea where the other components are installed at - and then reinstall the SharePoint components using the corrected path.
The SharePoint source and destination components should now appear in the toolbox for the data flow task. You can add the source and destination components to the data flow of the package.

Q: When using the SharePoint Credential, it works in Design but not at Runtime

This is because you are not using the default user and need to create a safe location to store the credential.
Check out the following site for how to store the username/pw so that you can run it outside of the designer: http://support.microsoft.com/kb/918760

Q: How do you suggest I use this in SSIS and execute it in my organization?

One way to setup this SSIS, is to use SQL Agent to schedule the job. However, make sure that the "Type" of the SQL Job Step is set to SQL Server Integration Services Package. After doing so, you will soon find that the package is executed using the account running Sql Agent. That can be changed doing the following steps:
1. Open SQL Server Management Studio and connect to your SQL Server
2. Under your Server -> Security -> Credentials; Add a new Credential with the login needed for your SharePoint.
3. Under SQL Server Agent -> Proxies -> SSIS Package Execution; Add a new Proxy and select the Credential you just created in the field marked "Credential Name".
4. Open the SQL Job Properties and on the step which runs your SSIS Package, the dropdown will now have the Proxy you created to run the Job under a different user account.

Q: Is it possible to make an ssis variable and pass this to the SITEURL in the SharePoint List Source?

Yes - please see: http://sqlsrvintegrationsrv.codeplex.com/discussions/225681

Pulling Data from SharePoint

Q: Using the SiteListViewName, I still see all of the columns of the original list

This is because views only filter the data rows from the backend that we're querying against, they do not filter the column data. If you want to remove columns from the Adapter, you can use the advanced tab and simply remove them. See the "Selecting Columns" Section of the MSDN Instructions.

Q: Wow, this adapter is SLOW to transfer my data! I Selected the source listview and am not happy.

Default usage selects every column in SharePoint. Often, most columns are extraneous or not necessary for your ETL. Those can be removed, see the "Selecting Columns" section of the MSDN Instructions.
Secondly, you can also have multiple SharePoint Sources in a dataflow pointing to the same list, bringing different columns (and the ID) and join them up using SSIS to parallel load lists with many columns.

Writing Data into SharePoint

Q: When updating my SharePoint List, every row is marked as modified, is there anything wrong with this?

Technically, no. However, it means the job will run slower as all of the list data is being sent to SharePoint to update each time. A more efficient way, since you are already updating particular Row IDs (if updating) is to use the SSIS components to filter and pass only the changed rows into the Destination Component. That will provide a more specific update and speed up the entire process significantly.
Here is one MSDN article on using the split function for doing this: http://msdn.microsoft.com/en-us/library/ms137886.aspx

Q: Why are columns not showing up in the Destination Adapter, when there are numerous fields in the List?+

The destination columns are filtered to excluded fields SharePoint has identified as readonly or hidden.

Q: What Datatype should I Use?

The easiest way to find out what to use to put things in SharePoint, is to use the source adapter and take a sample row out of SharePoint.
- Number Columns: Any Float type, Any Integer Type, or Currency (based on your data)
- Date/DateTime Columns: DT_DBTIMESTAMP, UTC format is assumed.
- Yes/No Columns: Single Byte Integer or DT_BOOL
- Text (Single Line): DTWSTR or DTNTEXT
- Text (Multi-Line): DTWSTR or DTNTEXT (SharePoint auto-formats this to HTML)
- Choice Columns: DTWSTR or DTNTEXT: This is a coded string which begins and ends with ;# Values are strings and separated by ;# Example: ;#First;#

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Sat, 10 Sep 2011 07:21:08 GMT

SharePoint List Adapter Changelog

9/10/2011
1) Updated installer to fix a problem to enable installing on a machine with SQL 2005 BIDS only (without Integration Services feature itself selected).

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring

Updated Wiki: SharePoint List Adapter Release Notes

kevinidzi — Tue, 30 Aug 2011 07:49:17 GMT

SharePoint List Adapter Changelog

8/29/2011
1) Expanded the OpenTimeout, CloseTimeout and SendTimeout in the WCF proxy to prevent some of the timeout issues reported.
2) Added support for using a Currency datatype when writing to SharePoint
3) Added ability for user to change External Field and Output Column to be a NTEXT for Choice Columns which are larger than default size.
4) Added mapping information to Documentation page for what datatypes are needed to get data into SharePoint.

8/4/2011 (Update for Deployment)
1) Fixed issue with prior installer, and also merged the SQL 2005/2008/Denali installers together. SQL will be auto-detected and found as needed.
2) BREAKING change to the code released on 8/3, I'm terribly sorry - please revert to your packages from before 8/3 release and use the 8/4 moving forward.

8/3/2011 (Moving Beta code to Release)
1) Replaced the Windows Installer, please report any issues. The target path will now attempt to be detected, to reduce installation errors.
2) An installer has been added compiled against SQL Server Denali CTP3, for those who are on the bleeding edge.
3) Deployment Solution used to cross-compile all versions has been added to source control.

6/6/2010 (BETA)
1) Added a beta version which includes a connection manager (Add a connection manager -> New Connection Manager -> Type=SPCRED). This will be required for all source/destination components, but once you create it, it can be shared.
2) Fixed issue with the Friendlynames for columns not coming through.

6/2/2010
1) Prior binaries are marked as such "Prior*" from the 3/18/2009 drop.
2) Added a deployment diagram
3) Patched some outstanding issues:
#12023: Changed Recursive parameter to RecursiveAll to allow traversing folders
#10018: Added condition for booleans in destination adapter
#9764, #12013: Added a T in the middle of the ToString("u") date to replace the space.
(internal): Increased Timeouts of WCF Proxies
4) Exposed the Attachment URL for list items
5) Added Credential support to SharePointUtility (not in Adapters yet, but uses the SharePointUtility).

3/18/2009
1) Hotfix update.

3/17/2009
1) Fixed support for multi-line text to be retrieved from SharePoint as NTEXT (there was a bug in previous attempt at this)
2) Added ability for user to rename source/destination columns and have the names remain even after refreshing the sharepoint list from the source (to pull in new columns or something)
3) Updated it so that numeric values from SharePoint will come into SSIS as null if they are null, vs being translated to a 0
4) This release will fix an issue with prior release's version # making it a breaking change for SSIS Packages

2/11/2009:
1) Support wide lists with lots of data in a column.
2) Support multi-row text columns up to 2 bil chars
3) Added Culture to allow proper formatting of numbers/dates on external systems in different languages
4) Changed validation so that if a column is ADDED to sharepoint, it sharepoint, it will not cause a validation error, only if a needed column is removed (helps with lists where people are adjusting them after you sync a process to them)
5) Fixed bug in datatyping where the data was being written using the wrong datatype (R8 Issue)

Also - The SharePointUtility.DLL has been added as a separate download to facilitate using it to access SharePoint Lists API without requiring