RoboPowerCopy – A PowerShell based RoboCopy clone

UPDATE 2021-07-23: Because of the Codeplex shutdown I have archived the project here: https://github.com/ikarstein/robopowercopy

Today I want to announce my latest development project.

RoboPowerCopy (v0.1.0.0)

It’s a PowerShell 2.0 based clone of Microsoft’s famous RoboCopy tool.

Get it on Codeplex: http://robopowercopy.codeplex.com

RoboCopy tool is part of my daily work. At work and at home. – You all now it! I hope so Zwinkerndes Smiley

I spend the spare time of some weeks for this project and now it’s ready to be published as ALPHA version (v0.1.0.0). The development is still in progress!! – Hopefully there is somebody out there that will use it – beside me.

I’ve done this using the (also) famous PowerGUI (Community Edition) by Quest.

“But way” you may ask.

RoboCopy is an EXE file and it has closed source. We (the customers) are not able to change it’s behavior. E.g. I’m missing a “verify” feature for years. Some people say: “It’s a copy tool not a backup tool”. Okay. That’s right. But I want to be sure that the copied data is valid and identical to the source data.

Because of that I thought about the possibility to realize a RoboCopy clone with PowerShell. – So I did it now.

RoboPowerCopy is plain PowerShell. No assemblies or PowerShell extensions are needed. – It uses some inline C# code.

RoboPowerCopy should “understand” at least the commands (options) of RoboCopy. – Currently I’ve implemented a lot of RoboCopy’s abilities. Not all. But some new features on top, e.g. verify and “read-and-check-before-copy”. “Verify” uses .NET hashing algorithms to compare the written data to the source data. If the destination file already exists the “Read-and-check-before-copy” feature reads every portion of data from the destination file before it replaces the data portion on the destination file with the source file’s data. This should reduce the time that is needed to copy large files that have only a few binary changes.

As you may now .NET has no build in capabilities to handle with very long paths with more than 256 characters. I have developed a set of helper classes in C# that enable RoboPowerCopy to deal with very long paths. – This C# is included in the script an will be compiled at runtime into an assembly in memory.

Here are the abilities:

  • Long path support (!!)
  • Restartable mode using a temporary “copy header” at the end of the copied file.
  • Copy security: SACL, DACL, Owner
  • Copy file attributes and timestamps
  • Same option set as RoboCopy (not completely implemented)
  • Can be extended because it’s open source
  • Customizable
  • Pure PowerShell with some inline C# code
  • Compare file times as UTC
  • Modify it on any Windows Operation System >= Win XP / Windows 2003 Server with Microsofts PowerShell development tol Windows PowerShell Integrated Scripting Environment that is part of PowerShell.

These are the limitations:

  • No multithreading support
  • No “Job” files at the moment
  • Slower than RoboCopy (Because RoboPowerCopy is Powershell and/based on Managed Code but RoboCopy is native optimized code…)

The following options I have implemented (v0.0.1.0). There are some options that are not available in RoboCopy. These I’ve marked red.

OptionDescription

since Version

/ACopy only files that have the Archive attribute set

0.1.0.0

/MRemoves Archive attribute of source file.

0.1.0.0

/A+:[HRAS]Add this file attributes to destination file

0.1.0.0

/A-:[HRAS]Remove this file attributes from destination file

0.1.0.0

/ECopy subfolder, including empty

0.1.0.0

/SCopy subfolders that are not empty. – “Not empty” means: at least one file is exists and is not excluded by any exclude option

0.1.0.0

/PURGERemove files on destination that does not exist on source

0.1.0.0

/MIRMirror the directory tree, like “/PURGE” combined with “/E”

0.1.0.0

/COPYALLCopy all file info: Timestamps, Attributes, Data, Owner, DACL, SACL

0.1.0.0

/COPY:[DATSOU]Copy selected file info: D=Data, A=Attributes, T=Timestamps, S=DACL, U=SACL, O=Owner

0.1.0.0

/DCOPY:[T]Copy selected directory info: T=Timestamps

0.1.0.0

/CREATEDoes only create the directory structure. Files will have zero size.

0.1.0.0

/SECCopy files with security. Like “/COPY:DATS”

0.1.0.0

/LEV:nCopy only n directory levels. 1=Copy only the given root directory.

0.1.0.0

/ISCopy even destination file seems to be the same as the source file.

0.1.0.0

/RCBWDuring overwrite this methos reads a file portion of data of the destination file and compares them with the source files portion of data before writing it. This my be useful while copying large files that have only few changes.

0.1.0.0

/VERIFYReread written data and compare them to the source data.

0.1.0.0

/SECFIXReplaces the security info of all destination files even if they are equal to the source.

0.1.0.0

/TIMFIXReplaces the timestamps of all destination files even if they are equal to the source.

0.1.0.0

/MAX:nMax file size to copy (in bytes)

0.1.0.0

/MIN:nMin file size to copy (in bytes)

0.1.0.0

/MAXAGEUsed to specify maximum file age based upon “LastWriteTime”. That will exclude files older than n days/date.
(If n < 1900 then n = no of days, else n = YYYYMMDD date).

0.1.0.0

/MINAGEUsed to specify minimum file age based upon “LastWriteTime”. That will exclude files newer than n days/date.
(If n < 1900 then n = no of days, else n = YYYYMMDD date).

0.1.0.0

/MAXLADUsed to specify maximum file age based upon “LastAccessTime”. That will exclude files unused since n days/date.
(If n < 1900 then n = no of days, else n = YYYYMMDD date).

0.1.0.0

/MINLADUsed to specify minimum file age based upon “LastAccessTime”. That will exclude files used since n days/date.
(If n < 1900 then n = no of days, else n = YYYYMMDD date).

0.1.0.0

/XD:<directory> [<directory> […]]Exclude this directories. Wildcards can be used for name matching.

0.1.0.0

/XF:<file> [<file> […]]Exclude this files. Wildcards can be used.

0.1.0.0

/XJExclude Junction Points, Symbolic Links (directories), Hard Links (files)

0.1.0.0

/XJDExclude Junction Points, Symbolic Links (directories)

0.1.0.0

/XJFExclude Hard Links (files)

0.1.0.0

/VDetailed output.

0.1.0.0

/VV[+][:<file>]Verbose output. If no file name is specified the messages will be written to the Powershell host using “write-verbose”

0.1.0.0

/CHUNK:nSize of the size of a portion of data during copy. In bytes. Default: 10mb.

0.1.0.0

/ZRestartable mode. A “copy header” will be used at the end of each file during the copy process. The header will be removed after copy is finished.

0.1.0.0

/BYTESShows file sizes in bytes

0.1.0.0

/R:nMaximum retry count if copy fails

0.1.0.0

/W:nWait duration before next retry. (In seconds)

0.1.0.0

/256Disable Long Path Support. This will use standard .NET classes (FileInfo, DirectoryInfo) instead of the special classes for file and directory names longer than 256 characters.

0.1.0.0

/NPNo progress – does not display “%”…

0.1.0.0

Using this option (except the red once) with RoboPowerCopy and RoboCopy should have the same result in both tools.

This features of RoboCopy does not work in RoboPowerCopy in v0.1.0.0:

  • Copy directory attributes
  • Set “Encrypted” attribute on destination files
  • Set “Compressed” attribute on destination files
  • …some more

Once more: It’s “ALPHA” in v0.1.0.0! – Therefore I want to ask for your help! – Please help to improve RoboPowerCopy. The project needs people how test and develop the script.

Please contact me at the projects Codeplex page if you would to spend your spare time for RoboPowerCopy.

Problem and Workaround: Organization Browser Silverlight Web Part is empty on other site than “MySite”

At first: This is a common problem. Using the "Organization Browser" Web Part on another page than on the MySite host web application results in an empty view. This means, the "Organization Browser" does not have any content. – As I said: This is a common problem.

First step to fix this is to create a “clientaccesspolicy.xml” file in the IIS directories of the SharePoint Web Applications.

See this blog post of Adam Preston:

http://www.tcscblog.com/2011/04/11/using-the-sharepoint-2010-organization-browser-in-another-web-application/

BUT:

In my current case it remains empty!!!

I used Fiddler to analyse the problem.

The Silverlight App “Organization Browser” executes a Web Service request but the response is empty. Not like an error but the Web Service does not find any data for the given account. Please see this screenshot for the request and its response:

image

I modified the request in Fiddler and removed the claim info “i:0#.w|” in the request. – And now it works. The Web Service does respond correct data!!!

image

I checked the authentication mode of both sites:

The MySite Web Application uses “Classic Authentication” and the Web Application from within I call the Organization Browser App is “Claims Based Authentication”. This results in bad request data for the Web Service. The “Claims Based” Web Application sends the user name in “claim format” but the MySite Web App cannot handle it. So I have to migrate the MySite Web App to Claims Based Authentication.

For the Migration of the MySite Web App from Classic Authentication to Claims Based Authentication I’ve written this script:

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 


######################################################################################################

    
  $url = "http://personal.sharepoint.local"
  $webapp = Get-SPWebApplication $url -ErrorAction SilentlyContinue
  if( $webapp -ne $null) {
     Write-Host "Web Application: $($webapp.Url)"
  
     Write-Host "  Active Claim Based Authentication"
     $webapp.UseClaimsAuthentication = "TRUE"
     Write-Host "  Update Web Application"
       $webapp.Update()
     Write-Host "  Provisioning Web Application"
       $webapp.ProvisionGlobally()
  
     #Claims Migration

     Write-Host "  Set Authentication Provider"
     $webapp = Get-SPWebApplication $url -ErrorAction SilentlyContinue
     Set-SPwebApplication $webapp -AuthenticationProvider (New-SPAuthenticationProvider) -Zone Default

     Write-Host "  Migrate Users to Claim Based Authentication"
     $webapp = Get-SPWebApplication $url -ErrorAction SilentlyContinue
     $webapp.MigrateUsers($true)
  }

After that I realized that the personal site collection does not have correct Site Collection Admin settings any more: There the original “Classic Mode” users are registered not the “Claim” user (login) names.

I’ve written this script to fix this:

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 


######################################################################################################

    
  $url = "http://personal.sharepoint.local"

  $webapp = Get-SPWebApplication $url -ErrorAction SilentlyContinue
  if( $webapp -ne $null) {
     $sites = Get-SPSite -Limit all
     $sites | % {
         $site = $_
        if( $site.Url.StartsWith("http://personal.sharepoint.local/sites/domain_", [System.StringComparison]::InvariantCultureIgnoreCase) ){
            Write-Host "$($_.Url)" -ForegroundColor Green
            $site.RootWeb.SiteUsers | ? { $_.IsSiteAdmin } | % {
                $user = $site.RootWeb.EnsureUser("i:0#.w|" + $_.LoginName)
                $user.IsSiteAdmin = $true
                $user.update()
            }
        } else {
            Write-Host "$($_.Url)" -ForegroundColor Red
        }
     }
  }

BUT: It does not work eighter Trauriges Smiley!!!

It seems to be a known limitation of the Organization Browser not to work at “Claims Authentication” enabled Web Applications.

BUT: I could create a wolkaround for this!!!

You need to edit the page where you want to use the “Organization Browser” in SharePoint Designer 2010 in Advanced Mode. – In my case I created a new Page Layout for my page derrived from the Page Layout “Welcome Links – Table Of Content”. In this case I modified this custom Page Layout.

This is the JavaScript code including the Content Placeholder ASP.NET tag for the code:

<asp:Content ContentPlaceHolderID="PlaceHolderUtilityContent" runat="server">
    <script type="text/javascript">
        var oldCreateHierarchyChartControl = CreateHierarchyChartControl;

        function CreateHierarchyChartControl(parentId, profileId, type) {
            var i = profileId.indexOf("|");
            //alert(i);
            if(i >=0 )
               profileId = profileId.substr(i+1,profileId.length-i-1);
            //alert(profileId);

                    var initParam = profileId + ',' + type;
                    var host = document.getElementById(parentId);

                    host.setAttribute('width', '100%');
                    host.setAttribute('height', '100%');

                    Silverlight.createObject('/_layouts/ClientBin/hierarchychart.xap',
                                            host,
                                            'ProfileBrowserSilverlightControl',
                                            {
                                                top: '30',
                                                width: '100%',
                                                height: '100%',
                                                version: '2.0',
                                                isWindowless: 'true',
                                                enableHtmlAccess: 'true'
                                            },
                                            {
                                                onLoad: OnHierarchyChartLoaded
                                            },
                                            initParam,
                                            null);

        }
    </script>
</asp:Content>

I’ve inserted this JavaScript code that overrides a JavaScript function created by the “Organization Browser” SharePoint Web Control. – This customized function removes the “Claim part” of the user name that is send to the Web Server by the Silverlight Application.

NOW IT WORKS!!! SmileySmileySmiley – On the Claim Authentication based Web Application the “Organization Browser” can be used!!!