SharePoint on-line 2013 web service call to execute a CAML query against a list

Well, talk about a journey; If you will be doing any SharePoint on-line 2013 workflows, you are going to need to understand this.

These were my inspiration
Great tutorial
Great Fiddler reference

Without those two, I would still be there at Christmas.  This is my summary;

I am querying a list in a sub web called ‘Config’ for a list Called ConfigItems

First you need to grab the auth cookies, browse to https://yourSPonline.sharepoint.com

FIDDLER: Get ContextInfo to grab a form digest from: This is a GET
https://yourSPonline.sharepoint.com/_api/contextinfo”

User-Agent: Fiddler
Cookie: rtFa+sO9PIAAAAA==; FedAuth=77u/PD94bWwg==
Content-Type: application/json;odata=verbose;
Content-Length: 0
Accept: application/json;odata=verbose;
Host: yourSPonline.sharepoint.com

Now, examine the json, find formdigest
Add it to your headers as X-RequestDigest
X-RequestDigest: 0x445623A1758C83A5,25 Mar 2014 07:37:55 -0000
So now you have

User-Agent: Fiddler
Cookie: rtFa+sO9PIAAAAA==; FedAuth=77u/PD94bWwg==
Content-Type: application/json;odata=verbose;
Content-Length: 0
Accept: application/json;odata=verbose;
Host: dev3.sharepoint.com
X-RequestDigest: 0x445623A1758C83A5,25 Mar 2014 07:37:55 -0000

Now you can call list api’s – Get all items from a sub web called Config USE A GET

https://yourSPonline.sharepoint.com/Config/_api/web/lists/getByTitle(‘configitems’)/items”

or via CAML, this is a PUT

https://yourSPonline.sharepoint.com/Config/_api/web/lists/getByTitle(‘configitems’)/GetItems(query=@v1)?@v1={“ViewXml”:”<View><Query><ViewFields><FieldRef%20Name=’Title’/><FieldRef%20Name=’ItemValue’/></ViewFields><Where><Eq><FieldRef%20Name=’ItemGroup’/><Value%20Type=’Choice’>SomeWorkflow</Value></Eq></Where></Query></View>”}
or
https://yourSPonline.sharepoint.com/Config/_api/web/lists/getByTitle(‘configitems’)/GetItems(query=@v1)?@v1={“ViewXml”:”<View><Query><ViewFields><FieldRef%20Name=’Title’/><FieldRef%20Name=’ItemValue’/></ViewFields><Where><Eq><FieldRef%20Name=’ItemGroup’/><Value%20Type=’Choice’>None</Value></Eq></Where></Query></View>”}

 

 

SharePoint on-line : simple 2013 workflow to send an email

Struggled for a while with this.

  • Download latest SharePoint designer 2013 (some early releases were buggy I am told)
  • Apply any SharePoint designer updates (Had an issue with designer not able to load visio)
  • Make sure the “Acount” Designer is using is your 0n-line account not your desktop account
  • Connect to an site collection (Not your admin site collection :-))
  • Enable site feature “Workflows can use app permissions”
  • Grant app permissions in your site collection see this (http://msdn.microsoft.com/en-us/library/jj822159.aspx)
  • Create a (in my case) site workflow
  • Add an app step
  • Inside the app step Include a send email action, send it to yourself
  • Publish the workflow
  • Go to site settings -> Site content -> site workflows
  • Execute your workflow
  • Check your inbox of the email address specified in your Profile

That’s all folks.

 

Search fix (Simple single server farm only)

I am constantly having issues with three things on SharePoint 2013 farms

  • Search
  • UP Synch
  • Managed metadata

So to search.  Often following a reboot the search admin screen will show one component or another in a non ideal state.  Red cross, Yellow warning triangle etc.

googling will show you many and various ways to fix some / all or none of these issues, re create index, flush SharePoint Caches, re provision Search host, and the list goes on and on and on…

For times when all these fail me, I have crafted this simple script to surgically take out a broken component and replace it with a shiney new one.  It is intended for single server with simple search topology and if you use it you are responsible for ensuring the code is fit for ‘your’ purpose.

I think the script is self explanatory.

 

[cmdletbinding()]
param (
    [Parameter(Mandatory=$true)]
    $component # Crawl, Admin, Query, Content, Analytics, Index
)
 
write-host "ONLY USE ON A SIMPLE, SINGLE SERVER FARM,  IF THIS IS THE CASE, COMMENT OUT NEXT LINE AND RE RUN"
break
 
# Grab the Search Service Instance
#
$Sinstance = Get-SPEnterpriseSearchServiceInstance -Identity $env:COMPUTERNAME
 
# Grab active topology
#
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
 
# Create a clone of active to work with
#
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active
 
# grab component giving us trouble
#
$problem = Get-SPEnterpriseSearchComponent -SearchTopology $clone | ? {$_.Name -like "$component*"}
$problem
 
# Remove troublesome component from clone
#
if($problem.GetType().ToString() -eq "System.Object[]") {
    $problem | Remove-SPEnterpriseSearchComponent -Identity $_ -SearchTopology $clone -Confirm:$false
}
else {
    Remove-SPEnterpriseSearchComponent -Identity $problem -SearchTopology $clone -Confirm:$false
}
 
switch($component) {
    "Crawl" {
        New-SPEnterpriseSearchCrawlComponent -SearchTopology $clone -SearchServiceInstance $Sinstance
    }
 
    "Admin" {
        New-SPEnterpriseSearchAdminComponent -SearchTopology $clone -SearchServiceInstance $Sinstance
    }
 
    "Query" {
        New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $clone -SearchServiceInstance $Sinstance
    }
 
    "Content" {
        New-SPEnterpriseSearchContentProcessingComponent  -SearchTopology $clone -SearchServiceInstance $Sinstance
    }
 
    "Analytics" {
        New-SPEnterpriseSearchAnalyticsProcessingComponent  -SearchTopology $clone -SearchServiceInstance $Sinstance
    }
 
    "Index" {
        New-SPEnterpriseSearchIndexComponent  -SearchTopology $clone -SearchServiceInstance $Sinstance    
    }
 
    default {
        write-host "Dont understand" $component
    }
}
 
# Make clone active
#
Set-SPEnterpriseSearchTopology -Identity $clone
 
# Clear out Inactive topologies
#
get-SPEnterpriseSearchTopology -SearchApplication $ssa | ? {$_.state -ine "Active" } | Remove-SPEnterpriseSearchTopology -Confirm:$false

scripted “re creation” of User Profile Service app from powershell as spfarm account after deleting the existing UPSA.

Thanks to Brian Lala and AutospInstaller for the inspiration and start-process syntax.

  • Ensure UAC is OFF
  • Run a PowerShell window as administrator
  • Paste the script below into your.ps1 file and run it
$ErrorActionPreference="stop"
 
$script = {
Add-PSSnapin microsoft.sharepoint.powershell -ea SilentlyContinue
$ErrorActionPreference="stop"
$setNetbiosNames=$true
$prefix="hostname"
$tld="domain.dev"
$mshPort="80"
 
#
 
$upServiceAppName="User Profile Service Application"
$upAppPool="SharePoint Hosted Services"
$upDbServer="UPS"
 
$upProfileDBServer=$upDbServer
$upProfileDB=("$prefix" + "_Profile")
 
$upSyncDBServer=$upDbServer
$upSyncDB=("$prefix" + "_Sync")
 
$upSocialDBServer=$upDbServer
$upSocialDB=("$prefix" + "_Social")
 
$upSyncInstance="$prefix"
 
$upMySiteHostUrl=("http://$prefix" + "-mysites." + "$tld" + ":" + "$mshport" + "/")
$upManagedPath="Personal"
 
 
 
$upsapp = Get-SPServiceApplication | ? {$_.TypeName -like "User Profile Service Application"}
 
if($upsapp -eq $null) {
    #write-host "Delete pre existing User Profile timer jobs"
 
    #Get-SPTimerJob | where {$_.name -match "User Profile Service.*"} |  % {
    #    write-host "deleting timer job:" $_.Name
    #    $_.Delete()
    # }
 
    write-host "Create UPSA"
 
    try {
        $upsapp = New-SPProfileServiceApplication `
                    -ApplicationPool $upAppPool `
                    -MySiteHostLocation $upMySiteHostUrl `
                    -Name $upServiceAppName `
                    -ProfileDbName $upProfileDB `
                    -ProfileDbServer $upProfileDbServer `
                    -ProfileSyncDbServer $upSyncDbServer `
                    -ProfileSyncDbName $upSyncDB `
                    -SocialDbName $upSocialDb `
                    -SocialDbServer $upSocialDbServer
 
 
        if($setNetbiosName) {
            write-host "Enable NETBIOS domain names"
 
            $upsapp.NetBIOSDomainNamesEnabled=1
            $upsapp.update()
        }
 
        $upsapp
    }
    catch {
        write-host $_
    }
 
}
else {
    write-host "Pre existing User Profile Service Application"
}
 
if($upsapp -ne $null) {
    write-host "Create UPSA Proxy"
 
    $upsAppProxy = Get-SPServiceApplicationProxy | ? {$_.TypeName -like "User Profile Service Application Proxy"}
 
    if($upsAppProxy -eq $null) {
 
        try {
 
            $upsAppProxy = New-SPProfileServiceApplicationProxy `
                            -ServiceApplication $upsapp.Id `
                            -Name $upServiceAppName
 
            $upsAppProxy
        }
        catch {
            write-host $_
        }
    }
}
else {
    write-host "Pre existing UPSA Proxy"
}
 
 
}
 
 
 
 
 
# this runs the script defined above under spfarm user account
# orginally sourced from Brian Lala autoSP-Installer for the "Start-process" syntax, Thanks Brian :-)
# See Brian T if it does not work for you
#
$secpasswd = ConvertTo-SecureString "p@55w0rd" -AsPlainText -Force
$farmCredential = New-Object System.Management.Automation.PSCredential ("domain\spfarm", $secpasswd)
 
$scriptFile = "$env:TEMP\UPCreate-Script.ps1"
write-output $script | out-file $scriptFile
 
Start-Process  -WorkingDirectory $PSHOME -FilePath "powershell.exe" -Credential $farmCredential -ArgumentList "-Command Start-Process -WorkingDirectory `"'$PSHOME'`" -FilePath `"'powershell.exe'`" -ArgumentList `"'$scriptFile'`" -Verb Runas " -Wait
 
 
 
 
$msg = "UP SA Creation done`n`n"
 
$msg += "You need to start the UP Sync service in `"Services on a server`"`n"
$msg += "Create a Sync connection, if you get error in create, try to use a new name for connection`n"
$msg += "Ensure UP Service account has Admin and full control of UP SA`n"
$msg += "Ensure msh app pool account has Admin and full control of UP SA`n"
$msg += "Ensure app pool account has Admin and full control of UP SA`n"
$msg += "Ensure sp content account has Admin = read people data in UP SA`n"
$msg += "Ensure sp farm account has Admin and full control of UP SA`n"
$msg += "Ensure setup (you) account has Admin and full control of UP SA`n"
$msg += "Configure service application associations and ensure UP SA is associated"
 
$msg += "Run a full sync`n"
 
write-host $msg

SharePoint 2013 – Workflow – FBA – Journey (on-prem)

If you are getting these 🙂

Retrying last request. Next attempt scheduled in less than one minute. Details of last request: HTTP Unauthorized to http://devVM.domain.dev/_vti_bin/client.svc/sp.utilities.utility.ResolvePrincipalInCurrentcontext

You may need this

Setting up SharePoint 2013 workflow is documented all over the internet, not all on one page though 🙂

This is how I got it all to work

Start here http://technet.microsoft.com/en-us/sharepoint/jj556245.aspx

  • Follow religiously this set of videos
  • Once configured enable site feature ‘WorkflowServiceStore’ use powershell Enable-SPFeature WorkflowServiceStore -Url http://yoursite
  • Now in the UI activate feature on your site ‘workflow can use app permissions’
  • Now grant full control to workflow ‘workflow’ http://msdn.microsoft.com/en-us/library/jj822159.aspx
  • wrap any steps of your workflow which fail for prmission related stuff in an “App step”

Simple eh ?

I guess we owe this complexity to SharePoint online and Office 365 (The future don’t you know)

Managed metadata service – not permissions

Well, this is a good one

Can access Managed metadata service in Central admin -> Manage service apps but not from the site itself.

Try this

Set the app pool for the site temporarily to run as the Farm account
IISRESET
Load site term store manager, it should now load
now …
Switch the site app pool back to SPAppPool or whatever it was
IISRESET
Load site term store manager, it should still load

My explanation is, Black magic and voodoo 🙂

Build a SharePoint 2013 VM

Use SQL 2012 and Windows 2012 using Brian Lala autospinstaller

New VMWare workstation 8 Vm (8 gig ram and 60 gig HD)
Use Windows 2008 R2 template – enable video acceleration

Install W2k12 Server – Standard
Enable RDP, hereafter everything over RDP

Rename-Computer SP2013
Restart-Computer

Make IP address static

Install-WindowsFeature AD-Domain-Services -IncludeManagementTools
Install-ADDSForest -DomainName yourDomain.local
Reboot

Create user Account
Add user to Domain admins, Enterprise admins

Log off domain admin, log in as user
Confirm IE 10.0 can see outside world

Shutdown
Create Base OS and AD checkpoint

Power on
Login as user
Windows updates (7 updates)
Turn off UAC
Reboot

Install SQL 2012 (SP1)

Restart
SQL Config – All protocals enabled
Restart SQL svc
Windows updates (2 updates)
Reboot
Shut down
Create Post SQL checkpoint
DAY 2 (4 attempts on this)
=====
Power on
create AutoInstallerInput-SP2013.xml on shared drive from host machine ( I used CMSPUBLISHING#0 as portal app template)

Edit config.xml and put in correct PIDKEY
put SharePoint media here as per Brian Lala docs

CMD window as admin
wscript /H:cscript
net use Z: \\192.168.245.1\d$ /user:yourdomain\user
Z:
Z:\CommonShare\autospinstaller-v3\createserviceaccounts.vbs (ask me if you need this)
Move users using AD tools to Services OU
CD \CommonShare\autospinstaller-v3\SP\AutoSPInstaller
AutoSPInstallerLaunch.bat
Made server reboot to do UAC even though UAC was already off – prereq requires a reboot
AutoSPInstallerLaunch.bat
Made server reboot to do UAC even though UAC was already off – ANOTHER prereq requires a reboot
AutoSPInstallerLaunch.bat
– publishing site icons all showing ??? (Fixed by switching masterpage from seattle to oslo – leave system master at seattle)
– Create a non publishing site /sites/t1 icons fine here
shutdown
Checkpoint PostFarm

From Spence Harbar blog:
We need to grant the Replicating Directory Changes permission on the domain to the DOMAIN\spups account. This account will be used to perform the sync, it will not run any services or application pools.

Right Click the Domain, choose Delegate Control… click Next
Add the DOMAIN\spups account, click Next
Select Create a Custom Task to Delegate, click Next
Click Next
Select the Replicating Directory Changes permission and click Next
Click Finish

Skipped rest of Spences instructions as dont pertain to a domain controller, which this is (even though that is a bad thing)

 

Setting super user / super reader account

This is the resolution to some tedious event log errors, which should be resolved.

add-pssnapin microsoft.sharepoint.powershell -erroraction silentlycontinue
 
function SetSuperCacheUsers()
{
[CmdletBinding()]
param (
    [string] $webApp,
    [string] $prefix,
    [string] $superuser,
    [string] $superreader
)
 
    $wa = Get-SPWebapplication -identity $webApp
 
    $wa.Properties["portalsuperuseraccount"] = ($prefix + $superuser)
    $wa.Properties["portalsuperreaderaccount"] = ($prefix + $superreader)
 
    $wa.Update()
}
 
SetSuperCacheUsers  -webApp "http://demo.escape.com" `
                    -prefix "i:0#.w`|" `
                    -superuser "escape\SPObjCacheSuper" `
                    -superreader "escape\SPObjCacheRead"
 
write-host "Now you should ensure that the accounts you set here have appropriate access in web App User Policy"
write-host "in central admin, Full control for the Super user and Full Read for the super reader"

Update all SharePoint 2010 service account passwords from one script

Nice to do this in one script, take note of additional steps after the script has run.

You must of course set the values between ### YOU NEED TO SET THESE and
### STOP SETTING NOW to those appropriate to your farm.

You will note that this script allows for a typical set of farm accounts
as required by a Least Privilege Service Accounts setup. You may add modify
or remove accounts as you require.

add-pssnapin microsoft.sharepoint.powershell -erroraction silentlycontinue
 
function Set-AdUserPwd
{
[CmdletBinding()]
Param(
    [string[]]$users,
    [string]$password
)
 
    write-host "Change AD passwords for system accounts"
 
    $users | % {
 
        $thisAccount = $_
 
        write-host "`tChange password for" $thisAccount
 
        $oUser = [adsi]"LDAP://$thisAccount" 
        $ouser.psbase.invoke("SetPassword",$password)
        $ouser.psbase.CommitChanges()
    }
 
    write-host "All done."
 
}
 
function ChangePasswordForManagedAccounts()
{
[CmdletBinding()]
param(
    [string] $newPassword,
    [string[]]$accounts,
    [bool] $UseExisting
)
 
    $newPw = ConvertTo-SecureString $newPassword -asPlainText -Force
 
    write-host "Change Managed account passwords"
 
    $accounts | % {
        $thisAccount = $_
 
        write-host "`tChange password for managed account" $thisAccount
 
        if($useExisting)
        {
            Set-SPManagedAccount    -identity $thisAccount `
                                    -ExistingPassword $newPw `
                                    -UseExistingPassword:$true `
                                    -Confirm:$false `
 
        }
        else
        {
            Set-SPManagedAccount    -identity $thisAccount `
                                    -Confirmpassword  $newPw `
                                    -Newpassword  $newPw `
                                    -Confirm:$false `
                                    -SetNewPassword:$true
        }
    }
 
    write-host "All done."
}
 
function DefaultContentAccessAccountPassword()
{
[CmdletBinding()]
param(
    [string] $searchAppName,
    [string] $account,
    [string] $password
)
    write-host "Change Default content access account password for" $account
 
    $searchapp = Get-SPEnterpriseSearchServiceApplication -Identity $searchAppName
    $c = New-Object Microsoft.Office.Server.Search.Administration.Content($searchapp)
 
    $c.SetDefaultGatheringAccount(
        $account,
        (ConvertTo-SecureString $password -AsPlainText -force))
}
 
#
# ### YOU NEED TO SET THESE
#
$systemAccounts = @(
                    "cn=sql-svc,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SpUps,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPServices,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPSearch,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPObjCacheSuper,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPObjCacheRead,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPMySiteAppPool,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPFarm,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPContent,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
                    "cn=SPAppPool,ou=YOUR_OU,dc=YOUR_DOMAIN,dc=com",
)
 
$managedAccounts = @(
                    "YOUR_NETBIOS_DOMAIN\SPFarm",
                    "YOUR_NETBIOS_DOMAIN\SPServices",
                    "YOUR_NETBIOS_DOMAIN\SPAppPool",
                    "YOUR_NETBIOS_DOMAIN\SPMySiteAppPool",
                    "YOUR_NETBIOS_DOMAIN\SPSearch"
)
 
$searchAppName = "Search Service Application"
$farmAccount = "YOUR_NETBIOS_DOMAIN\SPFarm"
$defaultSearchContentAccount = "YOUR_NETBIOS_DOMAIN\SPContent"
 
$theNewPassword = "L3tM31n"   
 
#
# ### STOP SETTING NOW
#
 
clear-host
 
# Change system account passwords in AD
#
set-AdUserPwd -users $systemAccounts -password $theNewPassword
 
# Managed account passwords
#
ChangePasswordForManagedAccounts -newPassword $theNewPassword -accounts $managedAccounts -UseExisting $true
 
# Default content access account
#
DefaultContentAccessAccountPassword -SearchAppName $searchAppName -account $defaultSearchContentAccount -password $theNewPassword
 
# Farm account
#
#
write-host "Change farm account password" $farmAccount
 
stsadm -o updatefarmcredentials -userlogin $farmAccount -password $theNewPassword
 
# its all over now
#
write-host "Remember to edit SQL service startup account password in `"Services`""
write-host "Remember to Restart profile sync service on central admin in `"Services on a server`""
write-host "Check out if SharePoint tracing service is running as a system account in `"Services`" make it local system"
write-host "All done."