Post on 15-Apr-2017
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Please help us!Thank our Sponsors:
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Low Cost Governance with Microsoft Online ServicesLOW COST, RICH CAPABIL ITY & RESOURCE INVESTED
BY SCOTT MITCHELL
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Session Objectives and Takeaways• Demonstrate how to use automation for applying IT/Information
Governance• Discover the features and capabilities of Azure Automation• Illustrate the power of data driven automation using XML & PowerShell• Perform CRUD operations on SharePoint Data using the REST API• Give a working demo!
And… My main objective is to• Inspire you to create solutions based on these ideas
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
• Training programs are ineffective at driving policy compliance. People do not remember and abide by policies without continued awareness updates and retraining• There is no immediate feedback when policies are inadvertently
violated• Tools that monitor and include end user communication and
awareness features are expensive and require more IT energy to operate• Tools that apply controls are not flexible when exceptions are
necessary and systems become brittle and hard to extend
Current State Problem Statements
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Test Policy for Demonstration Purposes
ESTABLISHED POLICYPROJECT SITES ARE NOT TO HAVE SUBSITESPolicy Definition
Per PMO Office
Project sites should not be structurally modified such as renaming the status list, document library or adding subsites.Project sites have end user training and processes paired with the site itself that break down if process related or project related material are buried in subsites.
Policy Link https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?k=project%20site%20integrity
Exception Notice
Site Owners are notified of noncompliance
Remediation Steps
Site owner is tasked to place content in original locations and remove subsites.
• PMO claims that their business processes break down often when PMs alter site structures.
• IT does not have the ability to configure granular controls to manage this policy through permissions
• Training is costly and not effective• By the time problems are
detected rollups and planning metrics have already gone off track
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
CommunicationAutomation AssetsSharePoint Lists
InformationSourceData [List]-SourceMetadata-SourceMetrics
Policy Rules [List]-Expressions
($sourcedata.variable –operator ‘value’)
Owner Mailbox-Policy Exception Notices
-Policy Title-SourceLink
Check-SitePolicies [RunBook]
> For Each Site in SiteCatalog> Process All Site PolicyExpressions> Send Exception Message
Update-SourceData [RunBook]
For All Sites Create/Update Site Details Invoke Check-SitePolicies
Schedule : TwiceDaily_1100Hrs
Policy Log Mailbox-Copy of all notices
Solution OverviewSolution Overview
IT Department/Policies Pages -Policy Details
-search?policypages=policy title
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
• Add SharePoint PowerShell Support to Azure Automation Runbook• Download the SharePoint Online Management Shell• https://www.microsoft.com/en-us/download/details.aspx?id=35588• Next Place it in a ZIP file and upload to your runbook Assets.• All Client and Runtime Libraries are included• All Assemblies are loaded automatically. So there is no need to add the types.
Setup and Configuration
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
• Runbook Assets are available at runtime including uploaded modules• Initializing for runtime helps so that you can run the same scripts
locally• Schedules, Logs, Source Control are all available and easy to use
Azure Automation
$spurl = 'https://stmtrial01-admin.sharepoint.com'$spsite = 'https://stmtrial01.sharepoint.com'if($env:SESSIONNAME -eq 'Console'){
$creds = Get-Credential -UserName 'stmtrial01@stmtrial01.onmicrosoft.com' -Message 'Login'Import-Module Microsoft.Online.SharePoint.PowerShell$path = (Get-Module Microsoft.Online.SharePoint.PowerShell).ModuleBase#[Void][System.Reflection.Assembly]::LoadFrom("$Path/Microsoft.SharePoint.Client.dll")#[Void][System.Reflection.Assembly]::LoadFrom("$Path/Microsoft.SharePoint.Client.Runtime.dll")connect-sposervice -url $spurl -credential $creds$credential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($creds.UserName, $creds.Password)}
else{$creds = get-AutomationPSCredential -name 'stmtrial'connect-sposervice -url $spurl -credential $creds$credential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($creds.username, $creds.password)
}
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
What can runbooks automate?Runbooks in Azure Automation are based on Windows PowerShell or Windows PowerShell Workflow, so they do anything that PowerShell can do. If an application or service has an API, then a runbook can work with it. If you have a PowerShell module for the application, then you can load that module into Azure Automation and include those cmdlets in your runbook. Azure Automation runbooks run in the Azure cloud and can access any cloud resources or external resources that can be accessed from the cloud. Using Hybrid Runbook Worker, runbooks can run in your local data center to manage local resources.
From - https://azure.microsoft.com/en-us/documentation/articles/automation-intro/
Azure Automation
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
PowerShell XML Supportfunction Set-SPOListItem <#<listitem rootSiteCollection='https://stmtrial01.sharepoint.com' listName='SiteStatus' keyField='Title'> <fields> <field name='Title' type='TEXT'>Hello789</field> <field name='Owner' type='TEXT'>Momma Cass</field> <field name='IssueStatus' type='CHOICE'>Resolved</field> <field name='DateCompleted' type='DATE'>8/1/2016</field> </fields></listitem>#># POSTING LIST DATA TO SHAREPOINT #$listName = [string]$ListData.listitem.listName $list = Invoke-SPORestMethod -Url "$siteUrl/_api/web/lists/getbytitle('$listName')" -Method Get -Credentials $Credential $Uri = $list.__metadata.uri $ListType = $list.ListItemEntityTypeFullName $keyFieldName = [string]$ListData.listitem.keyField $keyFieldValue = [string]$ListData.SelectSingleNode("//field[@name='$keyFieldName']").innerText $keyFieldType = [string]$ListData.SelectSingleNode("//field[@name='$keyFieldName']").type $CamlQuery = [string]"{{ 'query' : {{'__metadata': {{ 'type': 'SP.CamlQuery' }}, `"ViewXml`": `"<View><Query><Where><Contains><FieldRef Name='{0}'/><Value Type='{1}'>{2}</Value></Contains></Where></Query></View>`" }} }}" -f $keyFieldName, $keyFieldType, $keyFieldValue $items = Invoke-SPORestMethod -Url "$uri/getitems" -Method Post -Credentials $Credential -RequestDigest $digest -Metadata $CamlQuery -XHTTPMethod Post if ($items.results.count -eq 1) { #Update Items with new values $itemUri = $items.results[0].__metadata.uri $ListXMLValues = $ListData.SelectNodes("//field") $jsonListItems = [string]"{'__metadata': { 'type': '$listType' }, " foreach ($item in $ListXMLValues) { $fieldname = [string]$item.name $fieldname = $fieldname.replace(' ','_x0020_') $jsonListItems += [string]"'{0}': '{1}'," -f $fieldname, $item.innerText } $jsonListItems += "}" $jsonListItems = $jsonListItems.Replace(',}','}') $result = Invoke-SPORestMethod -Url $itemUri -Method Post -Credentials $Credential -RequestDigest $digest -Metadata $jsonListItems -XHTTPMethod Merge -ETag '*'
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
PowerShell Mapping Fields and Insert/Updating Items $sitesdata = [System.Xml.XmlDocument]::new() $listitem = $sitesdata.CreateElement('listitem') $listitem.SetAttribute('rootsitecollection',$CatalogSite) $listitem.SetAttribute('listname',$CatalogListName) $listitem.SetAttribute('keyfield',$CatalogKey) [void]$sitesdata.AppendChild($listitem) $fields = $sitesdata.createelement('fields') [void]$sitesdata.SelectSingleNode("//listitem").AppendChild($fields)
foreach ($site in $sites) { #Makesure fields element is clear... $fields = $sitesdata.SelectSingleNode("//fields") $fields.RemoveAll() #Then add all relevant site fields... #Field names are case sensitive... Use matching case when mapping fields.
$field = $sitesdata.createelement('field') $field.SetAttribute('name','Title') $field.SetAttribute('type','TEXT') $field.InnerText = $site.url [void]$fields.AppendChild($field)
. . .
Set-SPOListItem -ListData $sitesdata -Credential $Credential }
• Programmatically build an xml representation of a list item.• This xml is passed to the
set-spolistitem cmdlet which makes it so.
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Dynamic Script Evaluating Policy Rules
foreach($sitestatus in $SiteStatus.results.GetEnumerator()){ $Notices = @() #Process all policy rules foreach($policy in $Policies.results.GetEnumerator()){ $sb = [Scriptblock]::Create($policy.ExceptionExpression) $Exception = $sb.invoke() if($Exception){ $notice = New-Object System.Object $notice | Add-Member -MemberType NoteProperty -Name 'Owner' -Value $SiteStatus.owner $notice | Add-Member -MemberType NoteProperty -Name 'Source' -Value $SiteStatus.Title $notice | Add-Member -MemberType NoteProperty -Name 'Policy' -Value $Policy.Title $notices += $notice } } #End For Each Policy if($notices){ $body="" foreach ($notice in $notices){ $PolicyTitle = $notice.Policy $Owner = $notice.Owner $sourceref = $Notice.Source $fromAddress = 'IT@stmtrial01.onmicrosoft.com' $subject = 'Automated Policy Exception Notice' $body += @"
. . . "@ $creds = get-AutomationPSCredential -name 'stmtrial' send-mailmessage -to $owner -from $FromAddress -subject $subject -body $body -usessl -port 587 -smtpserver 'smtp.office365.com' -credential $Creds -BodyAsHtml } } } #End For Each Site Status
#TODO : tokenize and validate expressions rather than accept any value from the outside. This is just a demo! I know the security guys are heating up right now because I just accepted foreign values in my code as dynamic script.
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Loose Coupling – Search Based Policy Links $body += @"<h1>Automated Policy Audit Notice</h1>Dear Information Owner,<p/>Please review this notice and take corrective actions in an effort to comply with established electronic data policies.<h2>Policy Detail</h2><blockquote>Policy Name: $policyTitle</blockquote><blockquote><a href='https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?u=https%3A%2F%2Fstmtrial01%2Esharepoint%2Ecom%2Fsites%2Fit&k=$policyTitle'>Policy Details</a></blockquote><h3>Information Source</h3><blockquote><a href='$sourceref'>Information Source Link</a></blockquote><h3>Information Owner</h3><blockquote>$Owner</blockquote>"@
'https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?u=https%3A%2F%2Fstmtrial01%2Esharepoint%2Ecom%2Fsites%2Fit&k=$policyTitle
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Solution Walkthrough and Demo
Azure Automation Jobhttps://portal.azure.com/
SharePoint Sitehttps://stmtrial01.sharepoint.com
Solution Demo
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Add SiteSharing Policy Exception to the Policy listhttps://stmtrial01.sharepoint.com/Lists/Policies Restricted Intranet Site Sharing($sitestatus.sharing -ne 'Disabled') -AND ($sitestatus.Template -like 'BLANKINTERNET#0')
Rerun the Compliance Script and notice the new email sent to:stmtrial01@stmtrial01.onmicrosoft.comhttps://outlook.office.com
Add a New Policy Expression
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Session Objectives and TakeawaysSession Recap• Demonstrated a practical automated audit solution for modern IT• Discovered the features and capabilities of Azure Automation• Illustrated the power of data driven automation using XML & PowerShell• Performed CRU operations on SharePoint Data using the REST API
And… now you are thinking…
I can replace all human contact with a Site and an Automation Job
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Additional Resources
Free Microsoft eBookAzure AutomationBy Michael McKeown
Don’t forget the Module Gallery. You may make requests.
Office Dev Centerhttps://msdn.microsoft.com/en-us/library/office/dn531433.aspx
Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!
Please Complete An Evaluation FormYour input is important!
You can access Evaluation Forms at:
http://TulsaTechFest.com
Fill them out!
You can win additional prizes!
Like a $50 Best Buy Gift Card!!
Winner drawn – Midnight, Sun Aug 7th!