Startseite > SharePoint Online, SharePoint Server, SkyDrive Pro > SkyDrive Pro | PowerShell Script checks and moves folders and files

SkyDrive Pro | PowerShell Script checks and moves folders and files


SkyDrive Pro, PowerShell script, checks and moves folder and files

SkyDrive Pro and PowerShell?

Yes, no discrepancy. I used PowerShell because the script should be customizable. But in turn. Again and again, users have problems with files that do not synchronize with SharePoint (online) or SharePoint on premise.  And the users do not know, how they should solve the problem. sometimes users want to move complex folder structures into a SharePoint library(see remarks on the end of this blog post). And Andre Kieft, Microsoft partner technical consultant from the Netherlands has brought me with his little script on the right track.

If you are looking for a german description, here it is

Appended [09.09.13] Andre Kieft wrote an excellent Blog post on technet

Appended [03.10.13] Due to changes from Microsoft on SharePoint Online, it is now allowed, to sync Files with the extension *.exe und *.dll. I have made these changes also on the Powershell-Script.

Most problems with folders and files that are added in the SkyDrive Pro folder and then cause a synchronization error, are the restrictions on the SharePoint Server 2013:

However, we must distinguish

SharePoint Online (Office 365) blocked file extensions  (19)
f.e:  .ashx, .asmx… .xamlx
SharePoint Server 2013 and SharePoint Foundation blocked file extensions (104)
f.e.. .bas,  .bat,… .wsh
File- und Folder restrictions not allowed characters and extensions (9+20)
f.e.: “&”,”%”, or “#”, …
length of folder or filename folder< 256 , file < 128

Depending of the SharePoint Server a user must look between 48 and 133 restrictions on each folder and filename.

  • Simple copying a file into the SkyDrive Pro folder
  • Or copy entire folder with subfolders and files contained therein.
    SkyDrive Pro synchronizes only.  But the user see sync errors. Microsoft is working on this problem on SharePoint (online), may be you will see in the near future , that it will be allowed to use the “&” character in a filename. But not right now

    Solution

    The PowerShell Script SDPMove.ps1 

    The parameters:

    InputPath the source-folder
    f.e: C:\Test
    OutPutPath the destination folder 
    f.e: “C:\Users\hb\SkyDrive @ myCompany”
    -SPOnline optional parameter
    if it is not specified the script will check against SharePoint Server on premise.
    if specified, the script will check against SharePoint Online
    f.e: –SPOnline
    -Fix optional parameter
    if not specified, the script will only check and make no change
    (but you may see some errors)
    if specified, the script will check, change and

    a) folders will be copied to the destination folder
    b) files will be moved to the destination folder. 

    f.e: -Fix

    -Show optional  parameter

    if not specified, the script will give you minimal messages in the PowerShell console.
    if specified, the script will give you all messages

    f.e:  -Show

    -Language optional  parameter

    if not specified, the script will give messages in german language.
    This time 2 languages implemented: German and English
    Parameter: German, English

    SDPMove, start without parameters

    When it is called, the source directory is processed hierarchically. Here the flow chart

     

     

    image_thumb[2] After the script has been started, the parameters will be checked and then all not allowed extensions will be loaded.
    image_thumb[3] First run: The folder structure will be checked
    image_thumb[4] In the Check Folders & Files part ,all not allowed extensions and characters will be checked and replaced.
    image_thumb[7] the checked folder Structure is written into the destination folder.
    image_thumb[8] Second run: all files are checked.
    image_thumb[9] the individual files are written into the destination folder.

    Special features:

    folder

    Invalid characters are replaced.
    Invalid extensions are replaced with the addition of – HBx. The x stands for a numerical value.
    Folder remain left in the source directory.

    files

    Invalid characters are replaced.
    Invalid file extensions are not replaced, the file will remain in the source directory.

    folder and files Is determined by a change, that a folder or file with the same name exits, the script add –HBx  to the folder or filename. The x stands for a numerical value.
    the script cuts too long folder and filenames.

    You may notice and the Script name SDPMove says: the files will be moved to the destination directory. You may do a copy of the source directory in advance.

     

    Here a small selection, which changes SDPMove does:

    Original Change
    Order&Bill.docx OrderandBilldocx
    Offer&Version&1.docx OfferandVersionand1.docx
    request#1.xlsx request1.xlsx
    report~.docx report-.docx
    Test.dll.txt no change, remains left
    try.asmx no change, remains left

    Download

    Here you can download the zip file with the script

    Unpack you the zip file and place SDPMove. ps1 for example in the folder C:\PowershellScripts

    Start PowerShell and navigate to the path C:\PowershellScripts

     

    Call

    InputPath

    We assume you have a folder containing a complex structure with additional folders and files: "C:\Test"

    OutputPath

    You want to bring the "C:\Test" folder and all subfolders contained in it and all files on SkyDrive Pro: "C:\Users\Benutzer\SkyDrive @ myCompany"

    Then the command is:

    .\SDPMove.ps1 C:\Test “C:\Users\Benutzer\SkyDrive @ myCompany” –SPOnline –Fix –Show –Language English

    Problems when you run? (Execution policy)
    look here and run Set-ExecutionPolicy RemoteSigned   (as adnministrator)

    Script-Changes

    You may do script changes. If you run it against a Sharepoint Server On Premise you have to it. Check the file-extensions with you SharePoint administrator. If he has some more blocked file-extensions, than you have to adjust Init_SPOnPremiseIllegal. The same procedure, if he has allowed some file extensions.

    Remarks

    SDPMove is suitable also for moving complex folder structures. Whether it makes sense to bring a previously existing file folder structure 1:1 for SharePoint, this is question, SharePoint consultants should get here first of all their consulting skills in the game

     

    Feedback and comments welcome

    1. Avian
      27. August 2015 um 12:27

      Hello

      I try to download the script from various location, gettting error that site is ight be temporarily down. If possible pls send script thru email on avi081265@yahoo.com

      Avian

    2. Bob
      12. Juni 2015 um 17:26

      Fantastic. I’ve fought with this script all day, tweaking it to do what I want (e.g. ignore & as SharePoint online now allows this) but KEPT getting fatal errors.
      At last tried it from a different PC and it works perfectly! My Powershell was out of date, or corrupt, or the .NET framework the same. Uninstalled and reinstalled and all is now lovely!
      Thanks – this is going to save me a packet, as we have just been quoted a vast sum for a nice piece of software to do the migration, and now I won’t need it!

    3. 21. Mai 2015 um 18:40

      This a great script I would love to be able to run it once and produce a report that would summarizes the number of issues in each category then lists all the files that would change.
      Can anyone help with this?

    4. Sasa Ilijasevic
      20. Mai 2015 um 11:20

      In order to avoid false alarms regarding for ‘Found an illegal extension’, I suggest changing code in lines 190, 195 and in 200 when matching file name with a member of the array with illegal extensions. As an example when file name ‘species of wasps.docx’ is matched to ‘.asp’ in the present script it will return ‘true’. That is because in RegEx dot ‘.’ is one of special characters. In order to avoid this, backslash ‘\’ character should be added to the beginning of the ‘.asp’ so that file name is matched with ‘\.asp’.

      • Simon Margen
        8. Juni 2015 um 17:57

        Hi Sasa, Can you please post a line with the new match code containing this change?

    5. Anonym
      27. Januar 2015 um 11:03

      Is there anyway for this script to produce a report only and not fix the errors?

      • 27. Januar 2015 um 11:16

        the answer is No: But feel free to edit the powershell script

    6. Fokel Fiebleman
      14. Januar 2015 um 20:48

      This script deleted all of my source files. Unbelievable.

    7. Alice
      24. November 2014 um 11:09

      Thanks, I really like this article that provides good information related to move files and folders to SharePoint Online. I tried this LepideMigrator for Documents (http://www.lepide.com/lepidemigratordocuments/) which helps to migrate crucial data from file server and exchange public folders into SharePoint and move the attributes of all file, email, folder etc. and manage the schedule jobs as per your accommodation.

    8. ds18s20
      16. November 2014 um 19:11

      Just to clarify; the One Drive Business Edition sync client MUST be installed on the computer where this script runs, correct? So the script does NOT talk to the One Drive cloud at all directly but rather it moves local files from one directory to another while checking for size and file names, correct?

    9. ShaneDT
      3. Oktober 2014 um 03:57

      The script creates continual errors and stops due to the percentcomplete and count commands, once the value for count is above 100 it generates errors constantly.
      If you comment out (by adding # at the start of the line) lines 96, 97, 113, 114, 138 & 139 the script will run through to completion. It still reports every single file as ‘Found an illegal extension’ but this appears just to be incorrectly reported.

      • ShaneDT
        3. Oktober 2014 um 03:59

        PS: Thanks for the great script :)

      • ShaneDT
        3. Oktober 2014 um 04:33

        Also if you # out lines 176 & 204, only change, illegal extension and error messages are reported back to PowerShell. Much better if you’re scanning thousands of files :)

        • 3. Oktober 2014 um 09:49

          feel free to do changes

    10. Anonym
      3. Oktober 2014 um 03:40

      I’m having the same issue as Jake running on Windows 7.
      I find if I just continually hit enter the script continues to run until it completes.
      I’ll try commenting out that line, but also getting it on line 97 char:27 (see below)
      I suspect this is the percentcomplete command, because it’s returning a value over 100 it keeps generating an error.

      + Write-Progress <<<< -Activity $LMsg.Get_Item("FolderCheck") -PercentComplete $Count
      + CategoryInfo : InvalidData: (:) [Write-Progress], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Writ
      eProgressCommand

    11. jake
      1. Oktober 2014 um 01:58

      I am having an issue with the script on 8.1 with the following options “–SPOnline –Fix –Show –Language English” I am getting a lot of illegal extensions that are clearly ok (.pdf). I think the problem is in the status update code. I am getting:

      Write-Progress : Cannot validate argument on parameter ‘PercentComplete’. The 346 argument is greater than the maximum
      allowed range of 100. Supply an argument that is less than or equal to 100 and then try the command again.
      At C:\Users\Jake\Downloads\SDPMove\SDPMove.ps1:139 char:98
      + … ercentComplete $Count
      + ~~~~~~
      + CategoryInfo : InvalidData: (:) [Write-Progress], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.WriteProgressCommand

    12. Anonym
      11. September 2014 um 19:38

      Same issues here. Running win 8.1 with the -SPOnline option.

    13. Joost
      12. Juni 2014 um 09:30

      I got the same problem as Paolo. Every file gives an illigal extension.

    14. Paolo
      31. März 2014 um 13:00

      Hi. The script does not work. I get “Found an illegal extension” for every file and/or folder.

      • 31. März 2014 um 15:12

        some questions: OS? SharePoint Online? SharePoint On Premise? with or without SP1. may you tell me a filename?

        • Paolo
          31. März 2014 um 17:39

          Hi. OS: Windows 8.1. SharePoint Online. A filename: “Paolo.doc”. Anyway, when I use I -show on the command line, I get “Found an illegal extension” for EVERY file and/or folder. Thanks.

        • Paolo
          1. April 2014 um 15:05

          I answered to your questions in a previous post, but now my post has disappeared… Have you fixed the issue?

        • 1. April 2014 um 15:12

          I have no time these days

      • Paolo
        5. Mai 2014 um 13:35

        Hi. Did you find the time to correct the issue? Thanks.

        • 10. Mai 2014 um 12:04

          you are the only one, who has a Problem. Therefore I Need more details

    15. barry41
      13. Januar 2014 um 12:10

      You can also use Long Path Tool for such issues, it works good!

    16. 19. September 2013 um 13:54

      How do I get the powershell script described above?

    1. 6. Juni 2015 um 15:34
    2. 24. Oktober 2014 um 03:14
    3. 10. Juli 2014 um 18:20
    4. 29. Juni 2014 um 15:19
    5. 4. November 2013 um 16:05
    6. 30. Oktober 2013 um 13:37
    7. 2. Oktober 2013 um 17:32
    8. 1. Oktober 2013 um 21:45
    9. 6. September 2013 um 10:38
    10. 30. August 2013 um 14:39

    Kommentar verfassen

    Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

    WordPress.com-Logo

    Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

    Twitter-Bild

    Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

    Facebook-Foto

    Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

    Google+ Foto

    Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

    Verbinde mit %s

    Folgen

    Erhalte jeden neuen Beitrag in deinen Posteingang.

    Schließe dich 133 Followern an

    %d Bloggern gefällt das: