From 20 to 2,000 engineers on GitHub
Azure, GitHub and our Open Source Portal — by Jeff Wilcox
A brief history of our GitHub presence
Today, Microsoft as a whole is embracing GitHub faster than ever and I’m personally very, very excited to be a part of this embrace that is happening. It’s great to be able to share some of my perspective and opinion with you now.
In 2011 we set out to open source a set of Azure libraries across many languages, ship a new cross-platform command line and PowerShell experience, and were working on preparing to ship Mobile Services. Once the projects were greenlit the team huddled with a group of lawyers, pulled out “Uncle Steve’s Amex”, appointed a set of GitHub administrators and got to work.
I imagine this is how nearly every large GitHub organization has started, too. It grows fast from there. GitHub is a rocking service, but it isn’t necessarily designed for scaling to thousands of organization members today: it supports that number of users, it’s just that many of the key tasks, such as inviting people to the organization, involve what is often a manual workflow. GitHub provides an API that can help, but getting from that to a useful experience turns out to be the trick today.
What are some of the questions that we would like to answer about our GitHub presence and our organization members? Here’s a sample:
•Who is this random GitHub user? Are they an employee?
•Is this person in our contributor license agreement (CLA) database?
•Does this person still work at the company?
•Should this person be a member of a team on GitHub?
Azure Open Source Portal for GitHub
The Azure Open Source Portal for GitHub is a web app that lets employees authenticate with GitHub, authenticate with Microsoft (via Azure Active Directory), create a “virtual link” of these identities, onboarding to our organization(s), and then help to manage certain tasks depending on employee role. The portal securely performs tasks on behalf of organization administrator accounts when properly authenticated and authorized.
The portal is designed to be run on one or more cloud servers. It relies on a cache to help with sessions and to reduce pressure on the GitHub API.
Any OS
As a Node.js app the portal can run in many environments. In my development environment I regularly develop on my Mac or use my Ubuntu workstation and have really enjoyed using Visual Studio Code in all its cross-platform glory.
Requirements
•Node.js LTS release (Node.js 4.2.1, 4.2.2 or newer)
•A Redis server
•Azure Active Directory (for corporate authentication)
•An Azure storage account (for table storage)
Though I will cover it later in the post, I’m very open to working with potential contributors to remove some of the opinionated technology and policy choices in the portal. Together we can build something great.
Open Source projects used
Front-end components used in the project include Bootstrap, jQuery, and several small jQuery plugins.
The following Node.js modules are also used:
•applicationinsights
•async
•azure-storage
•express
•jade
•moment
•node-uuid
•octonode
•passport
•passport-azure-ad
•passport-github
•redis
(Also, these dependencies have their own dependencies which are equally important).
I contributed to Octonode OSS
As part of the building of this portal, I’ve contributed to the Octonode module, a Node.js service wrapper for the GitHub API, created by Pavan Kumar Sunkara. Thank you for your great library! I was able to contribute a number of updates related to the October GitHub changes that happened relating to organization permissions management as well as adding endpoints that had not yet been needed by other users of the library. The library is a clean, simple CoffeeScript library.
Service Dependencies
The portal makes use of a number of Azure services:
•Azure Active Directory
•Azure App Service
•Azure Storage — Table Service
•Azure Redis Cache
•Azure KeyVault
•Visual Studio Application Insights
App Insights is optional, as is KeyVault.
You can bring your own app server and Redis.
Swapping out table storage with your own storage preference would probably take 30–60 minutes.
And if you have a Passport (Node.js module for OAuth libraries) module that you or your organization uses, you could swap that out pretty easily for AD.
What’s Next?
My personal GitHub wish list
In the chance that this post is honored with a reader or two from GitHub, I figure this is as good time as any to share my personal organization-related GitHub wish list:
•The ability to plug in third-party authorization endpoints or systems like Azure Active Directory (Enterprise does have more auth choices, but I am talking about the public GitHub experience where our outbound open soruce work often goes)
•The ability to lock down repo creation to just public repo functionality, allowing anyone to create a private repo, but not go public — we want ‘public’ to be a special thing!
•Allow Organization owners to require 2-factor authentication. Members who do not have 2FA should be enforced then by GitHub.
•Fine-grained org permissions: allow people to manage their own teams for repos (similar to Collaborators) out of the set of approved organization members, similar to how we have expanded the Team Maintainer ability within our portal, for example.
What’s Azure doing on GitHub, anyway?
If you’re interested, here’s a super small subset of the projects we are working on…
•docker client contributions for Windows
•moving Markdown article content from GitHub to appearing on azure.microsoft.com (many articles have a Fork button at the bottom — go contribute and help improve our docs!)
•SDKs for managing Azure services and resources across .NET, Node.js, Java, Python, Ruby
•Command Line Tooling for Windows, Mac and Linux across both PowerShell and the cross-platform CLI powered by Node
•Mobile Services SDK for building mobile apps for building great Windows Phone, web, iOS, and Android apps powered by Azure
•Media Services (cloud streaming, encoding, live events) SDKs
•continuous integration services for teams and cloud virtual machines
•Contribution License Agreement (CLA) processing
•OAuth integration for products like Azure Websites
•Samples, documentation and resources for projects and solutions
•waagent, for managing Linux provisioning and VM interaction with the Azure fabric controller
•Azure Machine Learning modules, applications and utilities for Azure ML studio
•iisnode, for hosting Node.js apps in IIS on Windows
Jeff Wilcox is a Principal Software Engineer at Microsoft on the Azure team.
Tweet to him at @jeffwilcox
Continue reading:
http://www.jeff.wilcox.name/2015/11/azure-on-github/
Check out the Azure Open Source Portal:
https://github.com/Azure/azure-oss-portal
