Building Real Solutions on Top of SharePoint Online

I spent a substantial chunk of 2016 building a nicely complex solution on top of SharePoint online, which is a component of Office 365. I wanted to share a few thoughts on that in case anyone else is looking to do the same or hear from those that have done something similar.

What Did We Build?

This solution replicated and expanded upon an existing business solution — “V1” — that provided a SharePoint portal facilitating easier access to a collection of Tableau reports. Tableau is a leader in the data visualization field but it’s not really great at providing a portal experience. SharePoint obviously is.

By “portal” i mean a kind of one-stop shop for finding, accessing, manipulating and saving these manipulated reports. It caters to infrequent users (those that run a report once a month and hence, never learn all the really cool dials, knobs and buttons that Tableau provides) as well as a smaller group of true power users.

In addition to extending the rudimentary V1 portal, we provided a good bit more, including:

• Personalization to enable report curators to target some reports and other content to particular user groups
• A highly relevant search experience, taking personalization into account
• Classic portal features, such as targeted news, carousels, discussion forum and help
• A sophisticated “story builder” that enables users to create multi-page stories that seamlessly incorporate text, headings, live reports and annotated screen shots of reports

This was a fun project (and continues to be fun as I and team continue to expand it in 2017).

Infrastructure & Devops

It was built on top of SharePoint online with a key and vital assist from Amazon Web Services (AWS). We used Angular 1.5 as a client side framework and TypeScript. I used Visual Studio so the TS transpilation was automatic for me. At least one team member used Visual Studio Code and he never seemed to quite get a fully automated experience going. This could have been achieved, he just didn’t feel like solving it, I guess.

We used Visual Studio Online as our source code repository and git for managing it. Git is pretty much built right into the UI for visual studio. I occasionally dropped down into command line for exotic operations when required, but that was rare.

We used gulp to handle source code deployments. I invested quite a lot of time in gulp in the early days of the project and it was very much worth it. Using gulp, I was able to automate the process such that I could make a change in my source, press save and then switch over to he browser and test. Gulp would detect the change to the source file (via a watcher), bundle it up with its related bundle, minify that bundle and upload it.

I never got around to using a linter and we didn’t do any automated tests. One of these days, though!

Lastly, the business solution would not have been possible with AWS. We had multiple business requirements that either could never be handled client-side or should not be handled. Our client is an AWS shop and they spun up an AWS instance for us that we used to build technical components that met those requirements. We used AWS for things like:

• Controlled emails. I don’t believe there’s a straight-forward way to send emails client-side, meaning there is direct SharePoint endpoint we can access from JavaScript. Instead, we created a tidy REST endpoint on the server for this.
• Generating high resolution screen shots. Tableau provides a client-side approach (as well as enabling a kind of hacky option which isn’t terrible) that enables end users to grab screen shots. This is critical for stories since if you’re telling a story about a particular set of data specific visualization of that data, your often want a screen shot. The client-side options provide decent quality, but there’s a server-side option that provides the highest quality. We needed AWS for this.
• Generating PowerPoints and PDFs. It’s technically possibly to create a PowerPoint and PDF using purely client-side scripting, but it’s pretty weak tea compared to what you can do with a server. AWS saved our butts on this one.
• Data Extracts. Like SharePoint, Tableau includes a bit of a tagging feature. In future, we plan to use the AWS service to periodically poll Tableau and extract this metadata so that it’s properly replicated on the SharePoint side.

As general statement on SharePoint online development, I’d say that’s there’s no easy way to build a truly complete solution without a server side component. AWS did that admirably well for us.

Areas of Interest, Lessons Learned and All That Good Stuff

Here are a few of my thoughts and impressions while working on the project

Search: We used display templates to render results specific to different kinds of content types. We have four total CT’s at the moment, although three of them are able to share the same template. If we’d been smarter about it, we could have probably gotten away with one.

I think that display template are pretty terrible, really. First off — writing JavaScript embedded as a comment in HTML? Really? It demos well when someone has a prepared script, but otherwise, it’s a pretty horrible way of working! Since they are comments, you get zero tooling support — no intellisense and not even any formatting help. Seriously, they are awful.

They do at least provide an avenue for customization that we really needed. So even though they are appallingly bad, at least we have the option.

In addition to making them pretty, oh so pretty, we also added some interesting business functionality, particularly for adding a particular search result a list of favorites. We store favorites in a custom list and provided a little heart symbol to control favoriteness. If the heart is filled in, it’s a favorite, if not, it’s not. Click on the heart to toggle state. I wrote about that in greater detail here; https://medium.com/@pagalvin/using-dynamically-created-angular-directives-with-sharepoint-display-templates-fb10ae8cf446#.ir4rzvkh4.

Client side development: Client side dev is hella fun to me :). It’s a big topic, but here are a few highlights:

• We invested a lot of time getting the devops and the minute-by-minute dev experience right. With that in place, I used windows 10 virtual desktops. I typically had three open: 1) Visual Studio, 2) a command window (using the excellent cmdr app) and 3) web browser with a bazillion tabs. The command window would be running my gulp watcher/bundler/deployer. My typical edit/save cycle was: 1) Make a change, 2) flip over and see the change deployed and 3) test in the browser. It was a lovely experience and such an incredible time saver.
• More on infrastructure — using gulp, we were able to maintain a separation between environments. This is the command: “gulp bootstrap-tasks-and-watch — deployTo [targetEnv]”. This enabled us to maintain three different environments — integration dev, Test and Prod. It also enabled us to each have or own dev site collection. It made spinning up a new environment pretty easy.
• Last bit on infrastructure — we never automated the deployment of content types, columns, lists, etc. I did, however, build an “expected configuration” function. The expected configuration is a JSON data source that lists out all of the expected lists, site columns, site content types, etc. It highlighted missing items and misconfigured items.
• TypeScript is awesome. Among many other things, it allowed me to build up a really great SharePoint list helper function. Using TypeScript generics, I was able to write a generic function that provided all the CRUD operations and strongly typed to boot. I expect to write up something on this soon enough.

Branding: The solution called for a highly branded experience. We tried hard at first to customize the UI via CSS and jQuery, but despite herculean efforts, this proved impossible. (How many times has that phrase, “herculean efforts,” been used when trying and failing to implement a business feature in SharePoint? :) ). I know there’s a lot of pressure to avoid custom master pages, pages layouts and such. However, if you accept that limitation, how do you do a good job branding? I don’t think you can. You will end up with a site that has users mumbling “it’s nice and everything, but it kinda looks a lot like SharePoint.” I’m personally sick of that particular albatross. Once we embraced custom master pages and page layouts, we quickly put together a user interface that matched the design that our oh-so-clever turtle-neck-wearing eyeglass-wearing user experience design people created for us. We were basking in the glow of happy users. We made them so happy, they were practically begging us to dial back the happy givingness.

We also automated the deployment of these branding assets. I wrote about that here: https://medium.com/@pagalvin/tip-deploying-branding-assets-master-pages-etc-to-sharepoint-online-bb4a17bb8b74#.a4q25osn5.

User Profile: We needed a way for end users to describe default values and interests. For instance, in which country are they based? Which operating companies are most important to them? Any SharePoint pro would immediately want to reach for the SharePoint user profile for such information. However, this client is a multi-national client and this application is just one of many use cases, site collections, etc. in the global tenant. We didn’t have access to the user profile and the app’s personalization requirements were too specific. As a result, we were unable to use the SharePoint user profile, so we created a custom list instead and added these app-specific user profile properties there.

Amazon: Although not part of the “main” part of the application, it nevertheless plays a crucial role that elevated the whole solution from “that’s a decent solution with caveats” to “that’s a great solution!” and no caveats. You simply cannot build fully functional solutions on Office 365 of a certain size and complexity without some kind of server component. You can build very good great solutions, to be sure. However, in order to cross that gap from “good, for being in SharePoint” to “first class application” you need a server side component. Amazon did the job for us and did it well.

In Conclusion

I hope this summary of the solution and some of info I provided was of value. If you have any questions or comments, please do use medium’s cool feedback tools. I’d love to get a better understanding of what others are doing.

</end>