PowerShell

How to Remove Duplicates items in SharePoint Online List and keep one by PowerShell ?

Posted on Updated on

Hello folks,

We have faced recently a critical situation with a customer:
He was running a certain PowerShell script to update a SharePoint Online list with Data, it reaches 7000 items, a certain issue occurred in the script, and he found that he as 20,000 items in the same list, 70% was duplicates.

He wants to solve the situation in a safe way with no risk, by removing duplicate items and keep the Original one.
To solve the situation I did the following, This solution can be applied on all SharePoint versions 2010, 2013, 2016, & SharePoint Online, as the script is written to work as Client Object Model.

      1. I took back-up from the List, if you use SharePoint On-premise version, you can take back-up of the site. If you are using SharePoint Online, you can use Sharegate, however this step is not mandatory.
      2. Export AllItems to Excel using the Excel action in the ribbon, and make sure the ID Column is part of the export.
      3.  Download and setup a free tool named Kutools for Excel from here , the page contains also description how to use it to mark the duplicated column.
      4. Now take of the IDs from the Excel, and paste it in a text file on your harddrive
        Ids
      5. Then run the below PowerShell Script
        Add-Type -Path "C:\Path\SPOnline\Microsoft.SharePoint.Client.dll"
        Add-Type -Path "C:\Path\SPOnline\Microsoft.SharePoint.Client.Runtime.dll"
        $siteurl = "https://tenant.sharepoint.com/sites/yourSite/"
        $UserName = "siteadmin@tenant.onmicrosoft.com"
        $SecurePassword = Read-Host -Prompt "Please enter your password" -AsSecureString
        $Credentials = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist $userName, $SecurePassword
        $SPOCredentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword)
        $context = New-Object Microsoft.SharePoint.Client.ClientContext($siteurl)
        $context.Credentials = $SPOCredentials
        $web = $context.Site.RootWeb
        $context.Load($web)
        $context.Load($web.Lists)
        $context.ExecuteQuery()
        $listTitle = "List Name"
        $list = $Context.Web.Lists.GetByTitle($listTitle)
        $Context.Load($list)
        $Context.ExecuteQuery()
        $TargetSites = Get-Content "C:\temp\List-of-Ids.txt"
        foreach ($targetSite in $TargetSites)
        {
        $id=$targetSite.Split(";")[0]
        $caml = @"<View Scope="RecursiveAll"><Query><OrderBy><FieldRef Name='ID' Ascending='TRUE'/></OrderBy><Where><Eq><FieldRef Name='ID' /><Value Type='Number'>$id</Value></Eq></Where></Query><RowLimit Paged="TRUE">1</RowLimit></View>"
        $query = new-object Microsoft.SharePoint.Client.CamlQuery
        $query.ViewXml = $caml
        $items = $list.GetItems($query)
        $Context.Load($items)
        $Context.ExecuteQuery()
        if ($items.Count -gt 0)
        {
        for ($i = $items.Count-1; $i -ge 0; $i--)
        {
        Write-Host "Deleted: " $items[$i].Id.ToString() -ForegroundColor Yellow
        }
        $Context.ExecuteQuery()
        }
        }
        #$context.Dispose()

     

    Now, the items will start to be deleted, you can change the PowerShell script to fits your need.
    This method is safe because you see exactly what do you want to delete before executing the deletion script

Advertisements

Apply SharePoint Online template using PnP PowerShell

Posted on

In this article we will define a template, and apply it for other SPO sites using PnP library
Please follow these steps :

    1. Set-up PnP module latest version from here https://github.com/SharePoint/PnP-PowerShell/releases
    2. Go to your SPO tenant, and create a SPO Site, you can also define (List structure, Theme, Pages, Content Types) because all of those will be part of the template.
    3. Make sure that the app catalog is activated on the tenant (There is no relation between app catalog and the tenant, but there is a bug found in January PnP package that required App catalog to be activated or a failure occurred)
    4. After having the site ready, run the below PowerShell command:
      $Creds = Get-Credential
      $SiteURL = "https://test.sharepoint.com/sites/demo1/"
      Connect-PnPOnline -Url $SiteURL -Credentials $Creds
      Get-PnPProvisioningTemplate -Out C:\temp\demo1Template.xml
    5. You should see a progress running for a while to export all settings.
    6. Please open the XML file at the provisioned path (C:\temp\demo1Template.xml), and have a look, it is also doable to do medications through the file, by adding items, or fields, but you need to be careful not to crap the schema.
    7. Now it is time to import it to the destination site, Create a destination site on SPO with a default teamsite template
    8. Apply the new template by running the below command.
      $DestUrl = "https://test.sharepoint.com/sites/destination1/"
      Connect-PnPOnline -Url $DestUrl -Credentials $Creds
      $template = Load-PnPProvisioningTemplate -Path "C:\temp\demo1Template.xml"
      Apply-PnPProvisioningTemplate -InputInstance $template

After this, refresh the destination site, and you should see the template applied.
I suggest to start first with a simple template, then apply more complexity to it. this approach should run smoothly also with SP2016, but I haven’t tried to import a template from SPO and import it to 2016 or vise versa

How to get warnings and alerts for your SharePoint Online Site storage limits ?

Posted on Updated on

SharePoint Online in Office 365 is allocated a quantity of storage that’s based on your number of users.
If you want to get a warning email when your site exceeds its limit, please run the below script using PowerShell, you need to make sure that the dlls are correctly referenced.


#Connect to tenant admin center using GA credentials
$username = ""
$password = ConvertTo-SecureString "" -AsPlainText -Force
$cred = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
Connect-SPOService -Url -Credential $cred
#Local variable to create and store output file
$filename = Get-Date -Format o | foreach {$_ -replace ":", ""}
$result = ""+$filename+".txt"
#SMTP and Inbox details
$smtp = ""
$from = ""
$to = ""
$subject = "Alert : PFA Site Collection Quota Usage details"
$body = "PFA quota usage details"
#Enumerating all site collections and calculating storage usage
$sites = Get-SPOSite -detailed
foreach ($site in $sites)
{
$percent = $site.StorageUsageCurrent / $site.StorageQuota * 100
$percentage = [math]::Round($percent,2)
Write-Output "$percentage % $($site.StorageUsageCurrent)kb of $($site.StorageQuota)kb $($site.url)" | Out-File $result -Append
}
#Sending email with output file as attachment
sleep 5
Send-MailMessage -SmtpServer $smtp -to $to -from $from -subject $subject -Attachments $result -body $body -Priority high

Upload file to SharePoint Online using PowerShell

Posted on

This script help you to upload your file or document to SharePoint library using PowerShell

  1. Download SharePoint Online client library from here , and install it. (If you already have it, please ignore this step)
  2. Create a folder named Temp on your C: drive, and put the Excel/CSV file in it
  3. Modify the parameters below and run the below script.


#Specify tenant admin and site URL
$User = "site.admin@tenantname.onmicrosoft.com"
$Password = "YourPassword"
$SiteURL = "https://tenantname.sharepoint.com/sites/site"
$Folder = "C:\Temp"
$DocLibName = "DocumentLibraryName"
#Add references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM
Add-Type -Path "C:\Program Files (x86)\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files (x86)\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.Runtime.dll"
#Bind to site collection
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User,(ConvertTo-SecureString $Password -AsPlainText -Force));
$Context.Credentials = $Creds
#Retrieve list
$List = $Context.Web.Lists.GetByTitle($DocLibName)
$Context.Load($List)
$Context.ExecuteQuery()
#Upload file
Foreach ($File in (dir $Folder -File))
{
$FileStream = New-Object IO.FileStream($File.FullName,[System.IO.FileMode]::Open)
$FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$FileCreationInfo.Overwrite = $true
$FileCreationInfo.ContentStream = $FileStream
$FileCreationInfo.URL = $File
$Upload = $List.RootFolder.Files.Add($FileCreationInfo)
$Context.Load($Upload)
$Context.ExecuteQuery()
}
#Upload completed

Get SharePoint Publishing Pages with PowerShell

Posted on Updated on

This code is to get all publishing pages that is created in the last 30 days.
It is compatible will all On-Prem SharePoint 2007, 2010, 2013 and 2016
Please open PowerShell-ISE and put the code in it. then click Run


$ErrorActionPreference = "SilentlyContinue"
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Publishing")
#Start-SPAssignment -Global
$newline = [Environment]::NewLine
$webAppURL="https://yoursharepointwebapplication.com"
$currentWebApplication = [Microsoft.SharePoint.Administration.SPWebApplication]::Lookup($webAppURL)
$sites = $currentWebApplication.Sites
foreach($Site in $sites)
{
if ($Site.ServerRelativeUrl.Contains("market")){
#$Site = New-Object Microsoft.SharePoint.SPSite($SiteUrl)
$PubSite = New-Object Microsoft.SharePoint.Publishing.PublishingSite($Site)
$PubPageCount = 0
$newPubPageCount = 0
$ImageCount = 0
$DocCount = 0
foreach ($Web in $Site.AllWebs) {
$PubWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($Web)
$Pages = $PubWeb.GetPublishingPages($PubWeb)
foreach ($Page in $Pages)
{
if($Page.CreatedDate -GT (Get-Date).AddDays(-30))
{
$newPubPageCount += 1
}
}
$PageCount = $Pages.Count
$PubPageCount = $PubPageCount + $PageCount
#$Images = $PubWeb.ImagesLibrary.ItemCount
#$ImageCount = $ImageCount + $Images
#$Documents = $PubWeb.DocumentsLibrary.ItemCount
#$DocCount = $DocCount + $Documents
$Web.Dispose()
}
$OutputList += $Site.Url + "," + $Site.Allwebs.Count + "," + $PubPageCount + "," + $newPubPageCount +";"
$Site.Dispose()
#Stop-SPAssignment -Global
}
}
$OutputList

Cancel Nintex workflow stuck Instance using PowerShell

Posted on Updated on

In this post we are going to apply a PowerShell script to cancel all stucked workflow in Nintex

This script is used to run for SharePoint 2010, SQL Server 2008 R2, but it still can be used for other versions after running it on Stage environment

Take a copy from the below script and run it on the SharePoint server that contains Nintex workflows

if( (Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null )
{
Add-PsSnapin Microsoft.SharePoint.PowerShell
}
$web = Get-SPWeb "https://Microsoft.com/Sites/YourSite";
$web.AllowUnsafeUpdates = $true;
$list = $web.Lists["Your WorkFlow Configuration list"];
foreach ($item in $list.Items)
{
foreach ($wf in $item.Workflows)
{
if ($wf.InternalState -eq "Completed")
{
# Nothing To Do
}
else
{
# Writes Workflow Status
Write-Host $wf.InternalState
#Cancel Workflow
[Microsoft.SharePoint.Workflow.SPWorkflowManager]::CancelWorkflow($wf);
}
}
}
$web.Dispose();