Free screenreader for SharePoint

I'd like to share a short post on this thing I made: a screenreader for SharePoint.

https://www.youtube.com/watch?v=rE9CzuWchCg

What you're seeing is a standard Communication site in SharePoint on which I deployed a JavaScript component that reads aloud any page in your site, automatically or via the audio player controls at the top.
This shows the extensibility of SharePoint and might inspire you to help build an Intranet that is accessible for everyone in your organisation.
I've used SharePoint Framework, Microsoft Flow and the Bing Speech API to build it.
The code including installation instructions can be found on my github repository.

Trying out style transfer with Azure Notebooks

Perhaps I'm late to the party but only just discovered the cool things you can do with a style transfer algorithm which you can try out on https://deepart.io/ and http://ostagram.ru.

Here's an example:

This is my dog after style transfer

After doing some googling it turns out that the machine learning model is public property and can be downloaded. The academics behind it are Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge by the way. I then found a blog that implemented it in a Jupyter notebook.

I wanted to find a way to use it in a web application of my own, so I tried to use Azure Machine Learning to run the code and create an API with it. I knew that Jupyter Notebooks were supported in Azure ML (although in preview).
After some trying out, I couldnt get it to work because I couldnt find versions of Pytorch and torchvision that worked with the version of Python that Azure Machine Learning uses (3.4.5).

I decided to give it a last try in Azure Notebooks. And it worked!
Turns out it uses Python 3.5 which Pytorch and Torchvision latest versions require, and the Notebook ran.

The cool thing about Jupyter notebooks is that you can mix all kinds of content like code, text, images and the output of the code. See the output of the last run of my notebook.
At the bottom you see the styled image. Too bad this is low quality, Azure Notebooks is not very powerful and it doesnt use a Graphical Processing Unit, which would be required to style high resolution images like it is done on http://deepart.io. Even with this low quality the algorithm takes 10 minutes to complete style transfer on one image.
Trying to get higher quality output results in extremely long processing times and errors due to RAM limitations. Being able to run it in Azure Machine Learning wouldnt have helpen anyway, since it also doesnt support running code on GPU.

If I would want to create a real Web App like deepart.io from this Notebook I would need to use one of the Azure Datascience Virtual Machines. Sadly, they cost about 30$ a day to run, which would max out my bankaccount pretty soon. So, not a viable option for a hobby project :-)

If you want to try out the notebook at the Pytorch blog then you need to take into account that the images you supply to it need to be perfectly square and they both need to be of the exact same resolution. But if you follow the blog and first try it with the two demo images picasso.jpg and dancing.jpg then the notebook will run in Azure Notebooks without a problem.

Run PowerShell on SharePoint from Azure Automation

Here's a quick instruction on how to run a PowerShell script in Azure Automation on a SharePoint site. Bonus: use the new preview action in Microsoft Flow to call the script.

Use case
If you need to change some settings on a SharePoint site it's easy to do this with PnP PowerShell. For example it could be that your newly provisioned sites need some updates that cant be done during the site provisioning proces because you dont use PnP Core, or not the latest version.
By calling an Azure Automation runbook from Flow you can even trigger the PowerShell script after your site provisioning process using Microsoft Flow has finished, and update the desired settings in the site.

You will need the following to complete this tutorial:
- SharePoint Online and an account that has at least site collection admin rights
- An azure subscription.
- Some knowledge of PowerShell is useful.

Let's go!

Start in Azure:
1. Provision Azure Automation in a resource group
2. Go to modules > gallery > Add SharePointPnPPowerShellOnline
3. Go to credentials > Add credential named SHAREPOINTADMIN and fill in your SharePoint online account username and password.

The account needs to have the permissions in SharePoint that the script(s) requires of course, but it doesnt necessarily need sharepoint farm admin rights.
Site collection admin rights are enough if you want to run the script on a single SC.

4. Add a Runbook and fill in this script. This is an example script that can update the sharing settings of the site. Save and Publish it. Use the test pane to see if it works.
Take note of the following lines:

$cred = Get-AutomationPSCredential -Name "SHAREPOINTADMIN";
Connect-PnPOnline -Url $siteUrl -Credentials $cred;

This gets the credentials you have just added to Azure Automation and uses the SharePointPnPPowerShellOnline commandlets to connect to SharePoint.
After this you can use any PnP PowerShell commandlet you want on that site collection.

In Microsoft Flow:
6. Open Microsoft Flow and create a new empty Flow.
7. Select the trigger you require. I'm using the simplest trigger which is Button for Flow.
8. Add an Azure Automation - Create Job action and select the AA account you have just used. Select the runbook and fill in the parameters.
9. Run the Flow and check in AA if the script has run successfully and in SharePoint if the settings have been applied.

PS:
The Create Azure Automation Job action in Flow is in preview. It doesnt work correctly 100%.
For example, you will often need to reselect the values you set in the action when you edit the Flow.
Also, the parameters of the Azure Automation runbook are not always fetched correctly. Save and leave the Flow, or reselect the runbook in the Flow action in order for Flow to show the correct input parameters.
Lastly, The Create Job action can't send boolean type variables correctly to the AA Runbook yet. That is why I've declared all my variables in the Runbook as strings or integers and cast them to booleans in the script itself.
This is a limitation of the Flow action which I assume will be solved shortly.
Dont forget to save and publish the Azure Runbook. If you change the parameters of the runbook you will need to edit the Flow and update the parameter values that you input.

Add the SharePoint PnP PowerShell module in Azure Automation

Add the credentials to SharePoint in Azure Automation 

Use the Create Job action in Microsoft Flow to call the script and pass the parameters.

Send Email using SharePoint and MS Flow without Exchange

In this post I will (concisely :-) describe how to send a mail to a SharePoint user from Microsoft Flow in case the standard MS Flow send mail action doesn't work for you. This can happen if you don't use Exchange Online yet. Not a very normal case probably, but hey, maybe it'll help someone.
This also works for external accounts.

First you need to add the users you want to send mail to in a user group on a SharePoint site. This is required as the webservice we'll be using gets the user account info from a hidden list on the site that should include the user and his or her email address. So add the account to the site. The user doesnt actually need any permissions on the site, but the SharePoint site needs to 'know' the user.

Now before we can create the Microsoft Flow, you'll need to setup a SharePoint App on the site that we'll use the credentials of to send the email from Flow. Because we're going to call a SharePoint API from outside SharePoint we need a valid access token to be able to do so.

The method to get a valid token in a Flow described in an older post of mine: Call any SharePoint REST API from Microsoft Flow. It is based on the article Access SharePoint Online using Postman that clearly explains how to register an App in SharePoint to get a client id and client secret.

So see those two articles on how to get the token. You will need to call a specific URL with the client id and client secret passed to it in order to get an access token back. If you follow the previous post you will end up with a Compose action that contains the token string.

The final action is a HTTP Post to:
https://yourtenant.sharepoint.com/sites/siteyouregisteredyourapp/_api/SP.Utilities.Utility.SendEmail

with headers:
Accept: application/json;odata=verbose
content-type: application/json;odata=verbose
Authorization: Bearer <add the output of the compose action that holds the token here>

and the body:
{
                "properties": {
                    "__metadata": { "type": "SP.Utilities.EmailProperties" },
                    "From": "jurgen.wiersema@domain.nl",
                    "To": { "results": ["jurgen.wiersema1@domain.nl"] },
                    "Body": "Boe!",
                    "Subject": "Test flow email"
                }

}

In order to mail external users, you need to add the User Principal Name to the To property in the JSON, e.g. "i:0#.f|membership|jurgen.wiersema_gmail.com#ext#@jwiersem.onmicrosoft.com". This account needs to have the external mail address set in its profile of course.

Additional considerations:
- The user needs to have been added to the SharePoint Site you call the SendEmail webservice on. You can automate this within MS Flow by calling a different REST API method, which is the EnsureUser method.
- If you want to re-use this Email users Flow functionality, then you can build the Flow as HTTP triggered. If you pass the JSON above into it, which you then use in the action to post to the webservice, then you can re-use this Flow in all your other Flows by using a HTTP Post action to the Re-usable Flow URL.

Get token HTTP call

Send Email through a REST call - 1

Send Email through a REST call - 2

Email received by external user

Store link to executed Flow in SharePoint

What if you had a SharePoint list and you trigger a Microsoft Flow when an item is created in the list.

This might happen hundreds or thousands of times a day/week.

But how do you find out if the Flow did what it had to when it ran on the item? You go to flow.microsoft.com of course. Then you find out which Flow it was by looking through the list of crypticly named Flows you have access to. Finally you find it and you start browsing through all the Flow executions. You open them one by one until you find the Flow execution that handles the specific newly created item you are interested in.

This is not a very user friendly way of integrating MS Flow in SharePoint you might say! Wouldn't you want a link to the Flow execution log right from the SharePoint item that triggered the Flow? (hint: yes, you want this).

With this MS Flow expression you can get a URL to the executed Flow:

concat('https://emea.flow.microsoft.com/manage/environments/Default-0e0c2c6b-835a-4d45-8a92-4fac0d3be692/flows/', workflow().name, '/runs/', workflow().run.name)

The first part is the URL to your MS Flow tenant, so this is different for you. You need to specifiy your own values for both the region (in this case EMEA) and the Default-GUID, but you can easily get this from the browser address bar when you are inside MS Flow. The workflow() expression holds some metadata of the currently running workflow, and it turns out workflow().name is the guid of the Flow, while workflow().run.name is the ID that identifies a single execution of the flow. See the Logic App Workflow Definition Language for more info.

If you create a SharePoint list, and then create a Flow triggered on item creation in that list, then you can use this expression to save back the link to the executed Flow to the SharePoint list item.

In this way you can go to the exact executed Flow from the SharePoint list item itself.

The Flow that is triggered by creating an item in SharePoint. It can then do a bunch of stuff but in the end you want to store a link to the executed Flow back to the list item.

The Flow has run and the link is stored, in this case in the title field.

When you follow the link you end up in MS Flow on the executed Flow that you're interested in.

Provision a Microsoft Flow to a new site

What if...
... you had developed a site provisioning process that creates a new SharePoint site based on a great, custom PnP template.
... all the sites are created identically or have certain identical list, columns, content types.
... the process starts from the creation of a new list item in SharePoint.
... it even reports back to the list to say that it has successfully finished creating the site!

Super sweet! All is well.
Or not? What if the client wants to have a Microsoft Flow (running on new item created in a list) in all of the newly provisioned sites? That smells trouble!

Guess what? Thanks to new Flow management actions in MS Flow it is now possible to build reusable Flows and have them listen to a new list in a newly created site!

Here's how to do it, in general steps. You'll have to adapt it to fit your unique case.

You already have a Flow that runs after the sites have been created, and it has access to the URL of the new site.
Now create the 'template flow'. This is the flow that will have to be deployed on all your new team/project sites.
Create a very simple Flow in order to follow this procedure without too much trouble. Call it something like ReusableFlowMaster

The 'template flow'. Starts when an item is added to a list.

This is a pretty bad example because it updates the same item it runs on, so it will loop a few times. Who cares! It's not about this Flow.

Now that you have a new Flow, you probably think: This Flow holds a unique URL and unique GUIDs to the list it runs on. How are we going to make it reusable..?

From your main Flow that runs after site creation, we 're going to add a few Flow management actions and instantiate a new Flow (based on the ReusableFlowMaster flow) that is connected to a freshly created site.

1. Add an action and search for Flow.
2. Select the action Get Flow and select the ReusableFlowMaster
3. Create a new Compose action. Set it to the Flow definition that comes from the Get Flow action.
4. Now we're going to replace all the URLs and Guids that the Flow depends on so it points to a different list in a different site.
5. Add a new Compose action and put in the expression:

replace(string(outputs('ComposeActionThatHoldsFlowDefinition')), 'originalUrl', 'newSiteUrl') where originalUrl is the site the ReusableFlowMaster works on, and newSiteUrl is the URL to your newly created site. Note that we're casting the Flow Definition to a string and than replacing the old url with the new url.

6. Add a new compose action and put in the expression:

replace(outputs('New_Flow_Definition_-_replace_URL'), toLower('originalListGuid'), toLower('newlistGuid'))

where originalListGuid is the GUID to the list the ReusableFlowMaster works on, and new Guid is the Guid to the new list in the New site. How to get that is out of the scope of this article, doh!

7. Add a new Compose action and parse the previous actions' output to JSON:

json(outputs('YourFlowDefinitionStringWithAllUniqueThingsPointingTotheNewsite'))

PS: Don't give the actions such long titles :-)

8. I'm not sure this is necessary but i'm doing it anyway. Initialize a new variable of type Object and set it to the output of the JSON Parse Compose action. Call it something like FinalJsonObject

9. Now add a new Flow management action - Create Flow. Fill in the following:

Flow display name:  concat(body('GetReusableFlowTest')['properties']['displayName'],guid()). This is an expression, you can select the first part of the concat method by seleting the Display name property of the Get Flow action. You're appending a guid here to the title of the flow in order to make it unique.

Flow definition: Put in your variable FinalJsonObject

Flow state: Select started because you want it to do something right!?

Connection references: Select the button that says you want to input an array. Select the same connections that you got from the Get Flow action.

10. You're done!

11. Run the flow.

12. It Fails!!! Oh noes! The error is something along the lines of 'request is invalid and could not be serialized at line 4 column 1929'.

Ok, what you have run into is an issue with the MS Flow GUI. The way it has saved the variable with the FinalJSonObject does not work with the Create Flow action. Luckily a certain John Liu already found a solution.

What you need to do is export this flow to a zip file. Unpack the zip and in it (a few folders deep) is a definition.json file. Open it and search for @{variables .

You will find, for example: "@{variables('FinalJsonObject')}"

FinalJsonObject is the variable name you chose to put in the Create Flow parameter Flow Definition.

This syntax is incorrect. It needs to be like this "@variables('temp')" so without the accolades {}

Save the file and put together the zip file again in the same way as you unpacked it.

Upload the zip to Flow and give the Flow a new name. This one should work!

You can also still edit this flow, but I assume you should not touch the Create Flow action anymore otherwise you'll have to take the steps of manually editing the flow definition again.

Your main Flow will now generate a new, unique Flow based on a 'template' and attach it to a freshly created site!

Word of warning: Seriously test this solution if you want to use it, and think about how you would do about updating all the instances of the ReusableFLowTemplate once you've created dozens of Flows connected to new sites. I just did this as a Proof of Concept, it's not a solution of industrial strength.

Also, in case you want a more extensive Flow in the site, take note that anything unique in the Flow related to the site, URLs, lists, etc will have to be replaced by the actual values for the new site you're connecting to. If all your sites are provisioned with PnP a lot of the Guids of content types and columns will already be the same in each site. but for example list Guids are always unique. You will need to save the ReusableFlowMaster to JSON and check out the contents to see what you have to update in it.

Offline SharePoint Online site!

What I've shown in the video is how you can use a SharePoint site to serve as an mobile app that works offline!

The magic happens when you first enter the SharePoint site (and have authenticated on it): a piece of code is registered that makes the website available offline (a so called serviceworker). Next to this the data (that is stored online in Google Firebase) is synced to your device as well.
Now you can go offline and still use the App/site. I've built it so you can add reactions to a vacancy which are stored locally on your device. When you go back online, these reactions are synced to a SharePoint library.

So what's the point you might ask. Well, setting up a SharePoint site to be a real (single page) App allows for a great user experience on mobile and desktop with which you could let users file expense reports, book holidays, search for colleagues by expertise, etc. etc. For now, the new modern SharePoint experience does not support full page apps or customizing page layouts/masterpage so it works fast, has full branding and has an optimal user experience. By overriding the normal SharePoint experience with a SPA all this is possible without any modification to SharePoint except setting the homepage to an html page and uploading some html, js and css files.

Using this technique of registering a serviceworker to allow offline use of a website has the following pros:

• Your SharePoint App is usable offline.
• You can use sharepoint capabilities like identity, profile, lists, search when online and sync offline data to the site when going back online.
• You can design a great looking App on SharePoint that works perfectly on desktop and mobile phone.
• It's a website sothere's no need to put your App in the Google or iOS stores.
• You can provide access to the App to your whole organization, or to specific users. And also to external users through external sharing functionality in SharePoint! Everyone you want can enjoy the App.
• If you decide you do want a full native App on Android or iOS, the code is easily packaged as a real App.

This solution also has the following downsides:

• You cant serve the manifest.json that is required for a full Progressive Web App with SharePoint (it doesn't serve JSON files at all). No manifest means no custom splashscreen, custom icon, or automatic install popup when you first enter the site. So basically you can only get half a PWA.
• The App is a website and so does not have access to more advanced functionality of your phone. If you want better integration on a phone you would need to include those capabilities in the App and package it as an Android and iOS App. 
• Offline mode only works in Chrome and Firefox (or any browser that supports serviceworkers). Microsoft and Apple are still working on this feature for their browsers. 
• You will probably want to build this as an SPFx webpart later. For now full page apps arent yet possible in SPFx. PowerApps is another possibility to support offline use of SharePoint, but it has it's downsides as well.
• When you host this App on SharePoint anonymous access is not possible.

This App was built using Ionic Framework, which is a framework that is build on top of Cordova and allows you to build phone apps in Angular. It uses SharePoint and Google Firebase as backend. Next to this SP PnP JS is used to easily post data to and fetch data from SharePoint. An Azure Function was created to fetch the vacancies from a third party API and store them in Firebase. Firebase is not necessary of course, you could do everything with just SharePoint. But eventually this App will not be hosted on SharePoint :-)

The following 'tricks' were needed to get this working:

Ionic PWA

See a previous post of mine that details how to build a Progressive Web App using Ionic2.

Upload files

Use the trusty old Internet Explorer and Explorer view in SharePoint to upload your Ionic app's www folder to the root folder of your site. NB: If you want to use a different SharePoint site than your O365 root site, you would need to make all the links used in your app server-relative, i.e. /sites/yoursite/index.html.

No Cors fetching of files to cache

Turns out that the default serviceworker included with the Ionic templates gives an error when deployed to SharePoint. This is because fetching the files required for offline use from SharePoint results in a few redirects that eventually gives an error because of Cross Origin Request Sharing (the redirects go to different domains like microsoftonline.com). See this stackexchange post for my solution.

Online / offline detection 

I had some trouble correctly implementing the detection of going online or offline. The events would fire multiple times. This was due to the place where you registered for the events in your code and because Ionic seems to create multiple instances of pages. See this stackexchange post for my solution.

Let me know what you think about this post!

Website with App experience using Ionic

We may already be in the heydays of mobile Apps, but you will still run into companies that want an App-experience for their (internal) products. For the cases in which you don't need any native phone capabilities and don't want to add an App into the Google or Apple stores there recently has been added (by Google) a new development paradigm called: Progressive Web Apps (PWA). PWAs are web sites (that run on the internet) that behave like a real App on a mobile phone (in the chrome browser on Android phones).

The Progressive Web App

.. is installed as an App so it shows in the Apps section of Android

.. has a nice icon on your android homescreen

.. starts with a splash screen

.. runs full screen.

.. can be used offline.

This blog demonstrates how you can build (from a sample solution) and deploy such a PWA to Azure websites. 

This has already been documented in other places better than I can do it so I will link to those places for further investigation, but write the steps to perform the deployment here aswell.
You can see the end result here https://jwiersem-ionic-conferenceapp.azurewebsites.net . If you open that URL on an android phone, you will get a popup allowing you to install the App on your phone.

I will use the Ionic framework sample of their conference app, Visual studio 2017, an Azure account and a recent Android phone in order to build and deploy a working (sample) PWA.

First, as written on http://ionicframework.com/getting-started/ :

1. Install NodeJS
2. Open a Command window and install cordova and ionic with the command npm install -g cordova ionic
3. In a directory of your choosing, create a new app with the command ionic start. This starts a menu that asks for a name for your project, fill it in and when you're asked which template to start from select the Conference app. This is a completely fleshed out and functioning ionic app.
4. Change directory into the folder you created for the App and test the app by running the command ionic serve. Or connect your Android phone via USB cable and run ionic cordova run android. The latter will run your app on your phone with one command.
5. Note the WWW folder of this app. All the necessary files are added here during building the project and the index.html can be opened in a browser and it will work as a desktop app.
6. Now you have a working cordova app. But we wanted a PWA! What to do?
7. In order to use your (well not your code, but you downloaded it fair and square!) code as a PWA, follow this article from Josh Morony.
8. Do this:
1. In index.html comment back in the script that loads servicework.js
2. Also in index.html, remove the loading of the script cordova.js.
3. Run the command: ionic cordova build --prod
4. And you're done!
9. But wait, didn't we need an Azure account for something? Yessir, we want to really test the PWA as a website in Azure. In order to deploy it easily and quickly we need to follow a few more steps.
10. Open Visual Studio. Then click File -> Open --> Website. Select the WWW folder of your ionic app.

11. You need to do one more thing in order to get your PWA live. Right click your www folder and select Add -> Add new item. Select Web configuration file here. In the web.config file (you should be familiar with that btw!) add the following section inside <configuration>

<system.webServer> 

    <staticContent>

      <mimeMap fileExtension=".js" mimeType="application/javascript" />

     <mimeMap fileExtension=".json" mimeType="application/json" />
     <mimeMap fileExtension=".woff" mimeType="application/font-woff" />

     <mimeMap fileExtension=".woff2" mimeType="font/woff2" />

  </staticContent>

</system.webServer>
You need to add this to the webconfig, otherwise the required files for your PWA will not be served by the Azure website.

12. Right click www again and now click Publish Web App and deploy to an Azure App service.

13. Your PWA is live! Visit the httpS URL with chrome on your Android phone, and you will get a popup asking it you want to install it as an App.

14. You now have a website with the added advantages detailed in the beginning of this article.

Full screen PWA on your phone.

NB: Once you installed the PWA on your phone and the delete it from your homescreen, you won't soon get another popup on the website. But maybe you want to demo the process to someone else later... To get the popup back follow these steps on your phone:

1.       Open Android Chrome.

2.       Go to chrome://flags/#bypass-app-banner-engagement-checks

3.       Click Enable

4.       Click Relaunch Now

PWA Popup in Chrome on Android phone

Azure Function with PnP - Missing Microsoft.IdentityModel.Extensions.dll

Let's say you want to build a Azure Function for site provisioning in SharePoint Online. You think, I could use the .Net PnP library for that! And you're right. However, I ran into the problem of a missing dll when trying to authenticate to SharePoint using App-only credentials.
It turns out the the Nuget packages for PnP are missing a dependency that is present on your developer environent but not with the Azure App service you deploy the Function to.
The file that is missing is Microsoft.IdentityModel.Extensions.dll.
This file is available in C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.IdentityModel.Extensions\v4.0_2.0.0.0__69c3241e6f0468ca

In order to get the Azure Function working, add this file in a folder in your Azure function solution. Then click on it and in the properties window select Embedded resource for Build action and Copy always for Copy to output directory. This will make sure the dll is copied into the bin folder of the App service and that it is loaded when the Azure Function starts.

Hopefully this article helps someone in fixing this issue when using PnP in Azure Functions!

SharePoint Online: New web service to scale images

I guess I was still in my wintersleep in Februari when this was released, but there is a new webservice available in SharePoint Online for rendering optimal-sized images for any device/screen resolution (image renditions don't work well for this )

You can use this in order to minimize the amount of bits sent to your users, especially for mobile users. Or you could use it to minimize the risk of terrible performance when a content editor adds a 6MB image to your site.

The location is https://yourtenantname.sharepoint.com/_layouts/15/getpreview.ashx . This location is virtual so the service exists in any site collection (NB. so also at https://yourtenantname.sharepoint.com/sites/yoursite/_layouts/15/getpreview.ashx )

If you read the following posts you can learn how to use it with a Path query variable pointing to the URL of the image or with three guids that identify the image file.

Image renditions available - this article also contains the available resolution codes.

Generate thrumbnail from URL in SharePoint

This is an example URL: https://MYTENANT.sharepoint.com/_layouts/15/getpreview.ashx?path=https%3A%2F%2FMYTENANT.sharepoint.com%2Fsites%2FJurgenCommSiteTest%2FSiteAssets%2FSitePages%2FTest-nieuws%2F627-large_9051a70d15cd9b67ba2097efb9e6c74c488d7267.jpg&resolution=6

Note the Path query variable that holds a URL encoded locator of the file and the Resolution which should be a value from 0 to 6.

So you can easily use this new webservice in your display templates or other (JavaScript) code to allow a user a fast download of an optimal size image.

Run asynchronous code on schedule with Azure WebJobs

This short post describes how you can have a scheduled Azure WebJob that runs an asynchrouns method.

The problem with this scenario is that the WebJob, which is just a regular console application, cannot have an asynchronous Main method. So you need a way around that. Next to that you would to be able to trigger that method with a schedule.

In my solution I use a Settings.job text file that is in the root folder of the WebJob (Set copy to out put directory to 'Copy Always' in the properties of the file in Visual Studio.
The contents of the file contain JSON with a cron expression like so:
{
    "schedule": "0 0 6/12 * * *"
}
This is set to every twelve hours starting from 6:00. So far so good, and nothing you probably havent seen before or in other blogs.

Now suppose you used the Visual Studio template to generate the WebJob Project. This creates the main Program.cs file and also a Functions.cs file. The Functions file is meant for you to register yor methods that serve as triggers. I tried using the TimerTriggered function like below, but the result wasn't that good. If you Run and block the JobHost in the Program.cs (as is done in the VS template), this means that your job will always show a running state and the timertrigger will go off, but only if it's within the time out period of the webjob which is 120 seconds. So that is a crap solution.

Instead, define a manual trigger with the tag [NoAutomaticTrigger]. We will call this method from the Main method. Notice that this method is asynchronous. I use code in there to fetch from a API and save back to a different service that I already wrote and that is asynchronous.

public class Functions
    {
        public static async Task TimerTriggered([TimerTrigger("0 0 6/12 * * *")] TimerInfo timer, TextWriter log)
        {
            await Functions.RunJob(log);
        }

        [NoAutomaticTrigger]
        public static async Task ManualTrigger(TextWriter log)
        {
            await Functions.RunJob(log);
        }

Now, code your Main method in Program.cs to call and await the method, using the JobHost.CallAsync method inside a Task.Run delegate.

class Program
    {
        static void Main()
        {


            var host = new JobHost();
            Task aTask = Task.Run(() => host.CallAsync(typeof(Functions).GetMethod("ManualTrigger")));

            aTask.Wait();          
        }
    }

The solution is to use the Task class to run and await the asynchronous method. With the JobHost.CallSync you can call the method in another class. Parameters like the TextWriter are passed by the JobHost.

Call any SharePoint REST API from Microsoft Flow

EDIT:
There is a much better way to call the SharePoint REST APIs now. A new action has been added to Microsoft Flow to do just that easily. See https://sergeluca.wordpress.com/2018/05/03/assign-unique-permissions-to-a-document-with-the-new-send-an-http-request-to-sharepoint-action-how-to-use-the-sharepoint-rest-api-in-flow/ for more information.

Case:
You want to do an action on SharePoint Online from a Flow. But the action, that is supported in the SP REST API, is not yet available as an easily configurable action inside Microsoft Flow.
So you think, I should be able to use the HTTP action for this.

Well, you're right!

But how do you then authenticate to SharePoint Online? I tried to get a token from the supported SharePoint actions in Flow but these turn out to have been *sanitized*, ie. not usable in other actions.

I tried a bunch of things including using the OAuth authentication setting on the HTTP actions. But that required filling in the rather meaningless (to me at least) parameter Audience. I suppose this should be the App ID URI of microsoft flow in your AAD Tenant, but I wasn't able to find it.

Thank god for this article that I (re)found after useless trial and error: Access Sharepoint Online Using Postman by Shantha Kumar

The article explains in great deal how to authenticate and perform a call to any REST API endpoint inside SharePoint.

This is the general gist:
1. Register an App in SharePoint through the pages /_layouts/15/appregnew.aspx and appinv.aspx
This will give you a client-id, client-secret and on the page /_layouts/15/appprincipals.aspx you can find your tenant id (the guid after the @ sign)
2. Perform a HTTP post to https://accounts.accesscontrol.windows.net/<tenantid_guid>/tokens/OAuth/2
with headers
 Content-Type:application/x-www-form-urlencoded
and in the body (remember the request is URL encoded, so it's just one big string, not JSON) a string like this:
grant_type=client_credentials&client_id=<apponlyclientid>@<tenantidguid>&client_secret=<clientsecret>&resource=00000003-0000-0ff1-ce00-000000000000/<yoursubdmain>.sharepoint.com@<tenantidguid>
3. Get the access token for the output of the previous HTTP action. Use a compose action, and enter something like this:
"@outputs('HTTP_2').body.access_token" where 'HTTP_2' is the name of your previous HTTP action with spaces replaced by underscores.
4.
Use the output of the HTTP compose (that contains the Bearer token) as input for any REST API call into sharepoint.
For example, Perform a HTTP Post to https://<yoursubdomain>.sharepoint.com/sites/Provisioning/_api/web/lists/getbytitle('Documents')/items(2)/roleassignments/addroleassignment(principalid=23,roledefid=1073741827)
with the following headers:
Accept:application/json;odata=verbose
Authorization:Bearer [OutputFromComposeAction]
This will add List item permissions on a list item (if you have broken role inheritance on this item already and are passing valid values for list item id, principalid and roledefid)

Here are some screenshots how to do that in Microsoft Flow:

Before

After

First look at SharePoint Framework

Background

In the history of development for SharePoint many different development platforms have been introduced over the years. Among these were full trust solutions, bin-folder deployments, sandboxed solutions, Client-side object model, the different models for building Add-ins and JavaScript in a script editor webpart. And shortly we will have yet another way to customize SharePoint On-premises and Online, the SharePoint Framework (SPFx)!

This new development model does not really bring new capabilities with it but is instead (finally) a real delivery platform for building JavaScript solutions in a structured way. JavaScript has always been present in Web development and has, in recent years, matured and even taken over server-side development through NodeJS. Frameworks such as AngularJS, KnockOutJS and React are standard tools for front-end development since the introduction of highly user friendly Single Page Applications. The SharePoint framework is aimed to bring SharePoint development into the realm of the ‘common’ web developer in order to greatly increase the number of developers that can customize the platform.

This document describes my first experiences and thoughts on the new development model and the repercussions of it for SharePoint developers.

Introduction

SPFx will be available on SharePoint Online in Q4 of 2016 and later be deployable to SP2016 through a Feature Pack. The introduction in SharePoint Online will go together with several other changes among which is the new responsive page model. The new page model changes the way a user can edit the content of pages, greatly improving the user experience because no page reloads are required. This is the way almost all big web platforms work and soon SharePoint will too!

Beside the responsiveness of the buttons and other controls to edit pages, the pages themselves will also be designed in such a way that they are mobile accessible. The pages will respond to the user’s device to show an optimized view of the page for that screen. Finally, the new page model allows for almost full-screen customization of the page, only a small portion is reserved for the ribbon and a side menu. This makes it easier for developers to make SPAs in O365 and increase the usability of SharePoint. SPFx WebParts will be available on the new page model, the older ‘classic’ type pages and a little later also on publishing pages.

The SharePoint framework webparts are built with a multitude of web development tools such as NodeJS and Yeoman, which will be discussed below. The tools can also minify and package the SPFx webpart as an .spapp package which is deployed to your farm via the App Catalog. Microsoft’s idea is that you deploy the source files of your webparts to a publicly accessible CDN location. This should make the files quickly accessible to end-users all around the world, and is a more maintainable deployment location than in Style libraries or masterpage galleries scattered throughout your entire SharePoint farm, where they might be deleted or corrupted by power users in the business.

The project templates for building a SPFx WebPart supports several JS frameworks (KnockOutJS and React) for development and more will follow. This nudges developers to start using a framework which will increase stability and maintainability of the code.

Microsoft mentions the following characteristics of SPFx:

·        It runs in the context of the current user and connection in the browser. No iFrames.

·        The controls are rendered in the normal page DOM.

·        The controls are responsive and accessible by nature.

·        There is a life cycle that the developer is involved in.

·        It’s not just render, but load, serialize _and _deserialize, configuration changes, etc.

·        It is framework agnostic – You can use any browser framework that you like – React, Handlebars, knockout, angular – take your pick.

·        The tool chain is based on common open source client development tools like npm, TypeScript, yeoman, webpack, gulp, etc.

·        Performance is key.

·        SPFx client-side solutions that are approved by the tenant administrators (or their delegates) can be used by end users on all sites – even self service created sites like teams, groups, personal, etc.

·        Can be deployed in both classic web part and publishing pages as well as the modern pages.

Tooling

The necessary tooling for building SPFx webparts is vast and for most SharePoint developers an entirely new world. However, it is the standard for common web development and will bring new people to SharePoint but also open up new opportunities for SP developers that are willing to learn something new.

First of all you will need NodeJS and its package manager NPM. Through the package manager you will need to install the project generator (Yeoman) and the task handler (Gulp). When you then install the Yeoman project template for SPFx webparts you will get the SharePoint workbench with it. The SharePoint workbench is a local development tool that you can use to locally develop and test a SPFx WebPart. In order to be able to deploy a SPFx webpart to Office365 you will most likely want to use an Azure storage account with CDN endpoint. Any CDN will do but the Yeoman project template includes the gulp tasks to deploy to an Azure storage account automatically. The dependence of SPFX Webparts on a CDN (and possibly API’s for advanced scenarios) may provide a boost to Azure usage for companies using Office365 which is a welcome side effect in my book.

When you have installed these prerequisites and scaffold a SPFx project you can start developing. Of course you can use Visual Studio for this, but actually lightweight editors such as VS Code or Sublime should be preferred. Besides the editors you better keep your command line open because that is the way to run and debug the code! Through the defined Gulp tasks in your project you build, run, package and deploy your project through command line options such as build, serve and package-solution.

SPFx development defaults to using TypeScript as the preferred JS dialect. It is probably harder to try and develop a SPFx webpart without TypeScript so this new development model is another big push of TypeScript by Microsoft. You are also encouraged to try to use the Office Fabric UI controls as much as possible. Office Fabric UI is a set of common HTML+CSS controls that you’ll often want to use on Office365. Next to this SASS (‘programmable’ CSS) is something you will want to try and conquer in order to speed up and structure the front-end development.

The included JS frameworks with the Yeoman template are React and KnockOutJS. More will follow.

Framework review

The previous section shows the great amount of new tooling that is required to be a successful SPFx developer. The reliance on open source, common web development tools and frameworks has SharePoint development enter a new chapter in its development history. The introduction of SPFx may beckon a whole new group of developers towards SharePoint, and likewise push the ‘old’ SharePoint developers into the Wild West of structured JavaScript development. It makes possible a more mature and professional way of developing client-side applications on SharePoint. It steers away from the practice of inline scripts on a page, and towards packages that can be successfully maintained and deployed.

Next to this, this new way of development should make it easier for web designers and developers to work together because of the reliance on SASS, the Office Fabric UI and editors like VS Code or Sublime. Prototyping apps is also faster and easier thanks to the SharePoint Workbench, making rapid development and design cycles possible. One possible downside to this is that this model is not very open to the power users/functional developers that are used to using the script editor web part and deployment into the style library. Because of the heavy reliance of SPFx on all the tools and frameworks it might be difficult for these types of people to step into SPFx development. The biggest threat to the success of SPFx (as with the other SP development models) will be that not all SP developers will want to adopt the model and will keep doing their JavaScript development in script editor webparts because it is the easiest path to delivery. This could mean a messy implementation and difficulty in maintaining Office365 solutions.

The user will likely also gain big from the SPFx framework and the new page model. Responsive and mobile accessible pages are a hard requirement nowadays and finally this will also be possible in SharePoint. Because also all of the new pages can be customized, designers and developers will be able to deliver completely unique experiences that are highly user friendly without going against best practices in SaaS/Office365 development. Next to this the new document library pages will also become customizable thanks to SPFx.

SPFx code should be deployed via Content Delivery Networks according to Microsoft because of the decreased loading times and because it facilitates the maintenance of the scripts. Also, the SPFx will allow easy integration with Microsoft Graph API’s and can also be used to talk against custom WebAPI’s that provide functionality that can only be achieved by server side code. For example, running in a different context than that of the current user or safe communication with on premises applications. Because of all this, SPFx may push companies to get Azure subscriptions or make more use of their existing subscriptions by building custom WebApis and integrating them with Office365.

Tips and tricks

When the SharePoint Framework and the new page model has been introduced in SharePoint Online and On-premises and you start a new project it is advised that you still build a development VM. Even though you can use a non-server OS and it uses only lightweight applications it will still be advisable to use a dev VM for each client. In this way you can be sure that it stays stable and that all developers work with the same version of Node, NPM, Gulp, Yeoman, the SPFx yeoman template, etc. It is easy to start developing on your own desktop, but when time progress and you get more projects using SPFx, you will start to be required to upgrade these tools and you may end up with old projects for other clients not working anymore. Besides creating a full fledged (probably Windows) VM, you can also use the popular Docker container software. Waldek Mastycarz has released a Docker image specifically for SPFx development. This is also an easy way to try out SPFx without installing all the prerequisites. Docker is a great way to virtualize an environment, especially when you can use a tried and tested Docker image.

In order to become proficient at using the new tools a developer has quite a steep learning curve ahead of him/her. It is advisable to get at least a working grasp of the node modules used, the gulp tasks, command line options and one or more of the supported JS frameworks like React or KnockOut. An explanation of the node modules included in the SPFx project template should be delivered soon by the MS team that is responsible for it. The Gulp tasks are also included in the project template, so get to know them and perhaps even learn how to add new, custom Gulp tasks yourself in order to speed up the development cycle. The SPFx also provides a great opportunity to get to know TypeScript and build more reliable JS solutions. Lastly, you will need to know what is in the Office Fabric UI pack in order to increase the maintainability of your designs.

In order to make full use local development cycle using the SharePoint WorkBench, it is advisable to include mock data in your SPFx webpart. In this way you can quickly show you app locally before it is on Office365 and iterate faster during development. Using mock data is done as such:

1.      Create models can serve to define the data structure of the data to show

2.      Create a MockHttpClient class

3.      Add GetMockData and GetRealData methods to webpart class

4.      Import environment type

5.      Add RenderDataAsync method that switches between getting mock and real data based on environment type and renders the data

6.      Add call to render data in the main render method (also add html container to provide location for rendering the html)

See http://dev.office.com/sharepoint/docs/spfx/web-parts/get-started/connect-to-sharepoint for in-depth guidance.

The following Gulp tasks are included in the Yeoman project template and can be found in the folder /node_modules/@microsoft/sp-build-web/lib/index.js

-        Build

-        Bundle (option –ship)

-        Deploy-azure-storage (uses /config/deploy-azure-storage.json )

-        Package-solution (option –ship uses /config/write-manifests.json )

-        serve

Use option –ship in bundle and package-solution to deploy to CDN.

SharePoint Framework webparts allows a user to fill in properties of the webpart in order to customize the appearance or function of the webpart. A SPFx webpart contains a manifest file to set the most appropriate default values for these properties. An attempt at making the operation of setting a web part property user friendly is to make the properties reactive to user input by default. MS went a little overboard with this, because this means every time you type, the code in the SPFx webpart get executed again. This is not very usable when the variables are used in web service calls, which will often be the case. Because of this you will almost always want to disable the reactive nature of the web part properties. This is done by overriding a property in your SPFx webpart’s code like so:

Set property disableReactivePropertyChanges to true (non reactive properties)

protected get disableReactivePropertyChanges(): boolean {

    return true;

  }

The yeoman project template for SharePoint Framework apps contains a great deal. Below I mention the most used files and their use.

What are the files in the yeoman template for?

·        IHelloWorldWebPartProps.ts (web part properties interface class)

·        HelloWorldWebPart.ts (the webpart itself inclusing render method, property pane code, import statements)

·        HelloWorldWebPart.manifest.json (default value of webpart properties)

·        HelloWorld.module.scss (styling for your webpart)

SharePoint Framework webparts allows the developer to make use of a handy pageContext variable that contains some useful properties shown below.

WebPart.context.pageContext variable

-        SPSite guid

-        SPWeb guid, absolute url, relative url, title, permissions

-        SPUser: email, displayname, loginname

Currently the actual properties of the (publishing) page do not seem to be included in this variable. It would be very useful for the developers when the current page’s properties could be easily read so they can be used in the code in order to make each page unique. For example, showing the date of publication, categories/tags of the page, related articles/people, etc.

Something to keep in mind when starting development is that the tools like Gulp and NPM are not very fond of a deep folder structure. So make sure you start your projects very close to the root folder.

When you want to try out SPFx right now, and this might occur after introduction as well if it is updated, make sure you’re using the most recent version of NPM, Yeoman and the Yeoman project template. Breaking changes may have been introduced in a newer version. MS promises that after SPFx goes GA they will not introduce breaking changes anymore. An example of an error you might get is the following. Instructions for updating your tooling follows.

Error:

Error loading debug manifest script. (Error: http://localhost:4321/temp/manifests.js did not call System.register or AMD define. If loading a global module configure the global name via the meta exports property for script injection support. Error loading https://localhost:4321/temp/manifest.js) Ensure loading unsafe scripts is allowed.

Update tooling:

npm update yo –g (update yeoman)

npm update -g @microsoft/generator-sharepoint (update yeoman sharepoint framework app generator)

yo @microsoft/sharepoint (new project)

gulp serve (run template project)

In chrome you’ll get security warnings. Go to advanced -> progress to site. SharePoint workbench inside Office365 will work again as well.

Conclusion

Hopefully this document has shown what a useful development paradigm the SharePoint Framework might become. Everything stands or falls with community adoption. However, SPFx might actually lead to new communities evolving around customization of Office365/SP2016. It may also provide new opportunities for old-school SharePoint developers if they adopt the platform. It will be exciting to see the range of solutions that will be developed using this new development model!