Copy Item
In this article
Overview
Use this Snap to copy items from a OneDrive or Shared Library (document library) location to another location in OneDrive or Shared Library.
Prerequisites
A valid application in Azure Portal with appropriate permissions.
A valid Microsoft OneDrive license.
Write access to both source and target locations (folders).
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations
Copying files larger than 10 MB using this Snap fails when you specify a new name for saving the file. This is an issue that Microsoft has acknowledged. The work around is to:
Copy your large files using the Copy Item Snap to a temporary location in your OneDrive, with the same name.
Rename and move it to your target location using the Move Item Snap.
When copying large folders/files (10,000 or more) in OneDrive, if there is an unexpected asynchronous call return from the API, the Snap doesn’t fail instead displays a warning message in the output:
An invalid status was retrieved from the asynchronous call: 'null'. Learn more:https://learn.microsoft.com/en-us/onedrive/developer/rest-api/resources/asyncjobstatus?view=odsp-graph-online.This results in the specific folder or file associated with the error not being copied, and the Snap continues to copy the remaining records. Therefore, we recommend you to verify the copied documents and check the warning message where you can view the uncopied items, if any. Learn more on how to do so using the following example pipelines:
Known Issues
Snaps in the Microsoft OneDrive Snap Pack have an in-built retry mechanism for failing connections and requests; but, a high volume of requests from a client might reach the throttling threshold. In such scenarios, the Microsoft Graph limits further requests from that client for a period of time and returns an HTTP status code 429 (Too many requests), thereby causing the requests to fail. Hence, we recommend you to keep a check on the number of requests emanating from your client instance.
The Snap copies the item to the root folder when you provide invalid Target Relative Path/ID (without root:/). However, the Snap works as expected (throws an exception) when you provide an invalid Target Relative Path/ID that begins with root:/. We recommend that you ensure that the Target Relative Path/ID always resolves to a valid folder.
Snap Input and Output
Input/Output | Type of View | Number of Views | Examples of Upstream and Downstream Snaps | Description |
|---|---|---|---|---|
Input | Document |
|
| A document containing the source and target details for copying the driveItem. |
Output | Document |
|
| A document containing the result of the copy operation. |
Snap Settings
Parameter Name | Data Type | Description | Default Value | Example |
|---|---|---|---|---|
Label | String | Copy Item | Copy Item | |
Source driveType | Drop-down list | Required. Select a drive type from the following list of permitted drive types in this drop-down list (to locate files/folders for the copy operation) and specify the corresponding details needed:
| OneDrive | documentLibrary |
Source User | String/Expression/Suggestion | Required for Drive Type options - OneDrive and All. Not needed for Drive and documentLibrary. Select a user from the suggested list of users to copy files/folders from that user's OneDrive location. The list of users displayed varies based on your access to their OneDrive locations. This option is meant for administrator users who are authorized to manage other users' drives, as well. | N/A | Snap Team:snap@snaplogic.net |
Source Shared Library | String/Expression/Suggestion | Required for Drive Type option - documentLibrary. Not needed for OneDrive, Drive, and All. Select a shared library from the suggested list of libraries to copy files/folders from that library location. The list of shared libraries displayed varies based on your access to them. This option is meant for users who have write access on both the source and target shared libraries. | N/A | NewSharedLibrary |
Source Drive | String/Expression/Suggestion | Required for all Drive Type options. Specify the name of the source drive to copy driveItems from. The suggestions for this field are based on the Snap account and the Source User or Source Shared Library selected. For the drive type Drive, specify/pass the ID of the source drive to copy the driveItem from, typically from the upstream Snap. For the drive type All, select a drive from the OneDrive or a Shared Library drive. Specify the Source Relative Path/ID of the driveItem, next. | N/A | OneDrive |
Source Relative Path/ID | String/Expression | Required for all Drive Type options. Enter the relative path for the driveItem to be copied, beginning with root:/. Alternatively, pass the item ID in this field. Relative Path and ID
| root:/ | /Pictures/Tour1/Image01.png |
Target Drive Same As Source | Check box | Select this check box to indicate that the source and target drive locations for copying the driveItem are same. If this check box selected, the Snap ignores the values in Target DriveType, Target User or Target Shared Library, and Target Drive fields and pastes the item in the path provided in Target Relative Path/ID. | Selected | Not selected |
Target DriveType | Drop-down list | Required. Select a drive type from the following list of permitted drive types in this drop-down list (to locate files/folders for pasting the copied driveItems in) and specify the corresponding details needed:
| OneDrive | documentLibrary |
Target User | String/Expression/Suggestion | Required for DriveType options - OneDrive and All. Not needed for Drive and documentLibrary. Select a user from the suggested list of users to paste files/folders to that user's OneDrive location. The list of users displayed varies based on your access to their OneDrive locations. This option is meant for administrator users who are authorized to manage other users' drives, as well. | N/A | Snap Team:snap@snaplogic.net |
Target Shared Library | String/Expression/Suggestion | Required for Drive Type option - documentLibrary. Not needed for OneDrive, Drive, and All. Select a Shared Library from the suggested list of libraries to paste the copied files/folders into that library location. The list of shared libraries displayed varies based on your access to them. This option is meant for users who have write access on both the source and target shared libraries. | N/A | NewSharedLibrary |
Target Drive | String/Expression/Suggestion | Required for all Drive Type options. Specify the name of the target drive to paste the copied driveItems into. The suggestions for this field are based on the Snap account and the Target User or Target Shared Library selected. For the DriveType Drive, specify/pass the ID of the target drive to paste the copied driveItem to, typically from the upstream Snap. For the drive type All, select a drive from the OneDrive or a Shared Library drive. Specify the Target Relative Path/ID of the driveItem, next. | N/A | OneDrive |
Target Relative Path/ID | String/Expression | Required for all DriveType options. Enter the relative path beginning with root:/, to paste the copied driveItem. Alternatively, pass the item ID in this field. Relative Path and ID
| root:/ | /Pictures/Tour2/ |
Target Item Name | String/Expression | Enter a unique name to save the file in the target location as, if you want to rename the file while pasting it. This works as expected only for files less than 10 MB. See Limitations for more information. To paste the file with the same name, leave this field blank. | N/A | Image01.png |
Number Of Retries | String/Expression | Specify an integer value to set the maximum number of reconnection attempts that the Snap must perform, in case of connection failure or timeout. | 0 | 3 |
Retry Interval (Seconds) | String/Expression | Enter, in seconds, the duration for which the Snap must wait between two reconnection attempts, until the number of retries is reached. | 1 | 2 |
Snap Execution | Drop-down list | Select one of the three following modes in which the Snap executes:
| Execute only | Validate & Execute |
Examples
Working with Drive Items (Folders and Files) in OneDrive
Microsoft's OneDrive and SharePoint dialects treat files and folders as drive items for all the similar operations (CRUD) they allow. Microsoft OneDrive Snap Pack provides multiple Snaps to perform these operations. This Pipeline demonstrates some of these operations - Creating folders, copying items, moving items, and deleting items. It assumes that you have configured your Azure application and the Snap accounts already. The Pipeline uses the following parameters.
We use the Create Folder Snap to create a new folder in the OneDrive (or Shared Library) location. Next, we use a Copy Item Snap to create a copy of an existing item in another folder/shared location. The Delete Item Snap removes an item from the source location. The Move Item Snap moves the copied item back to its source location. We also use a Mapper Snap to feed required information to the Move Item Snap.
Create Folder Snap |
|---|
|
Output |
|
Now, we copy the newly created folder (mentioned in Source Relative Path/ID field) to another target location (Target Relative Path/ID) using the Copy Item Snap.
Copy Item Snap |
|---|
|
Output |
|
Let us delete the original folder from the first source location using the Delete Item Snap. Notice the value/expression used in Source Relative Path/ID field in this Snap.
Delete Item Snap | Output |
|---|---|
| |
The output for the Delete Item Snap displays the summary of the delete operation. We use a Mapper Snap to return the name of the item and pass it to the Move Item Snap.
As the last step, let us connect a Move Item Snap to move the copied item (in the old target location) back to its original location (target location for the Move Item Snap). To do so, we capture this name of the item and pass it to the Move Item Snap as the source item detail.
Move Item Snap | Output |
|---|---|
| |
Upon completion of the task of moving items, the Snap provides the details of the driveItem that was moved to its original location.
Copy 10,000 documents using Copy Item Snap
The following example pipeline demonstrates how to copy a large number of folders/files (10,000 in this example). This example assumes you have configured the necessary accounts and prerequisites in SnapLogic.
Configure the Sequence Snap with 10,000 documents to generate a sequence of numbers from 1 to 10,000 to create unique folder names.
Configure the Create Folder Snap to create a new folder for each sequence number in the OneDrive location. On validation, the Snap displays the paths of the newly created folders with additional details.
Create Folder Snap Configuration | Create Folder Snap Output |
|---|---|
| |
3. Configure the Copy Item Snap to copy the specified documents to a target location. On validation, the Snap displays the paths of the copied items in the target location with additional details.
Copy Item Snap Configuration | Copy Item Snap Output |
|---|---|
| |
After copying the documents, you can connect any write-type Snap (File Writer or S3 File Writer) to Copy Item Snap to write the input data into a database. You can check if any warnings have occurred while copying the folders/files from the written data.
Check warning messages encountered while copying
The following example pipeline demonstrates how to read a JSON file containing details of folders copied and filter out any warning messages encountered during the copying process. This can help you verify the copied documents and check the warning message where you can view the uncopied items, if any.
Configure the File Reader Snap to read the onedrivePerfVerify.json JSON file, which contains details of the copied folders and any warnings encountered. On validation, the Snap displays the content of the binary output of the JSON file, showing details of copied folders and warnings.
File Reader Snap Configuration | File Reader Snap Output Preview |
|---|---|
| |
2. Configure the JSON Parser Snap to parse the binary data read by the File Reader Snap. On validation, the Snap displays the contents of the parsed JSON data as a document downstream for further use in the pipeline.
3. Configure the Filter Snap with the filter expression $warningMessage != null to filter out the JSON data to show only those that contain warning messages. This allows you to focus on issues encountered during the copying process. On validation, the Snap displays the filtered warning messages, detailing any issues encountered so you can take further action to troubleshoot.
Filter Snap Configuration | Filter Snap Output |
|---|---|
| |
Downloads
Important Steps to Successfully Reuse Pipelines
Download and import the Pipeline into SnapLogic.
Configure Snap accounts as applicable.
Provide Pipeline parameters as applicable.