Integrating GitHub Repository Information into a Website

GitHub provides a comprehensive REST API that you can call to retrieve information about repositories, issues, milestones, or any other information stored on your account.

Use GitHub’s project management features, and keep your website up-to-date automatically.

For the Riley Entertainment website, I’m using a private GitHub repository for this website. I’m keeping track of tasks and upcoming enhancements with issues and milestones, and pulling that information onto the home page in a panel to keep visitors up-to-date. I plan to do the same with the repository for my next game project, The Colon Case.

Keeping you up-to-date on how I plan to keep you up-to-date.

For those who prefer a more visual guide, I’ve put together a companion Youtube video to go along with this post.

TL;DR Version

If you’re technically savvy enough, here’s a summary:

1. Don’t bypass the Overview section. Read the entire Resources in the REST API article, as it provides details about required HTTP headers, authentication, and other important information.
2. Go through the Guides section, and test things out on the command line using cURL before jumping in to your own implementation.

This article will go over important details about the GitHub API, and provide example code written in PHP using the Client URL library.

Important Details about the GitHub API

You can see from the Reference section just how many features are provided by the GitHub API. I’m only accessing the Issues and Repositories APIs, but you could tap into any other feature you wanted.

I would strongly recommend starting out by testing the API using the command-line tool cURL, as suggested in the Getting started with the REST API guide. You can access the root API call at https://api.github.com/, which returns a JSON object listing URLs for all the available features.

These are the most important points for your implementation:

1. While you can access the GitHub REST API without authentication, you’ll not only be limited in what features are available, but also in how many calls you can make per hour. You can generate an authentication token by going to your user settings, then Developer settings, then Personal access tokens.
2. Requests must be sent with the User-Agent header set. It is suggested that you pass in your username, or the name of your application.
3. It is recommend that requests also send the Accept header, with a value of application/vnd.github.v3+json.
4. The API may respond with a redirect on any request.
5. To deal with rate limiting, the GitHub API allows you to send conditional requests with the If-None-Match or If-Modified-Since headers. The easiest way for an implementation to handle this is to store the ETag response header, and cache the most recent result of a specific call. Responses with a status code of 304 (Not Modified) do not count toward the rate limit.
6. For API calls that return a list of items, you can control pagination with the page and per_page query-string parameters. The response will include a Link header that you can use to determine the total number of pages of results.

The Client URL Library in PHP

I’m still fairly new to PHP, but it seems like the Client URL library is the best choice for PHP developers. I’m actually hoping someone from the PHP community will correct me on this, and let me know there’s a new Rest API library that’s a little bit cleaner to deal with… 🤣

While the file_get_contents function is convenient for simple calls, there are simply too many mechanics at play with the GitHub API for it to be reliable. You’ll have to deal with request and response headers, redirects, and a variety of response codes.

The GitHub REST API is only accessible from a secured URL. This means you’ll need a valid Certificate Authority to perform any API calls. If you don’t want to create your own certificate for your local web server, the cURL website currently provides the most recently updated certificate from Mozilla. In your php.ini file, you’ll need to uncomment the line curl.cainfo= with the path to the certificate file.

Performing a call with the Client URL library generally follows these steps:

1. Call curl_init to create a resource handle, which will be passed into all future cURL functions.
2. Call curl_setopt or curl_setopt_array with each option. PHP provides a large number of constants prefixed with CURLOPT_ to use as identifiers.
3. Call curl_exec to perform the API call.
4. Call curl_errno and curl_error to check for errors.
5. Call curl_getinfo with CURLINFO_RESPONSE_CODE to retrieve the response code.
6. Call curl_close to free the resource handle.

For the GitHub API, the most important options to set for the request with curl_setopt are as follows:

1. CURLOPT_URL, set to the URL.
2. CURLOPT_FOLLOWLOCATION, set to true to ensure redirects are followed. This can be coupled with CURLOPT_MAXREDIRS, if you want to set it to fewer than the default of 20.
3. CURLOPT_RETURNTRANSFER, set to true to ensure the response is returned as a string.
4. CURLOPT_HTTPHEADER, set to an array of strings containing the request headers.
5. CURLOPT_HEADERFUNCTION, set to an anonymous function to process the response headers.

Example PHP Implementation

I’ve organized the source code for my implementation as follows:

/source/php
  RileyGitHubApiConfig.php
  GitHubApiUtils.php
  GitHubRepoMilestonesApi.php
/public_html
  index.html
  /api
    /github
      get_website_updates.php
  /javascript
    SiteImprovementsPanelManager.js

The files in the /source/php folder are outside of the website’s public space to ensure they’re not directly accessible through the browser.

Here’s the source code for the PHP files. I think this provides a decent working example for anyone looking to do something similar. You can look at the source code on the Riley Entertainment website for index.html and SiteImprovementsPanelManager.js.

GitHubApiUtils:

<?php
namespace Riley\GitHubApi;
class GitHubApiConfig {
 public string $userAgent;
 public string $authorizationToken;
}
class GitHubRequestHeaders {
 public string $userAgent;
 public string $authorizationToken;
 public string $etag;
}
class GitHubResponseHeaders {
 public string $etag;
 public int $rateLimitRemaining;
 public int $rateLimitUsed;
 public int $rateLimitResetUtcEpoch;
}
class GitHubGetRequest {
 public string $url;
 public GitHubRequestHeaders $headers;
}
class GitHubCurlError {
 public int $errorNum;
 public string $errorMessage;
}
class GitHubGetResponse {
 public GitHubCurlError $curlError;
 public int $code;
 public array $headers;
 public string $body;
}
function createGetRequest(string $url, GitHubApiConfig $config): GitHubGetRequest {
 $result = new GitHubGetRequest();
 
 $result->url = $url;
 
 $result->headers = new GitHubRequestHeaders();
 $result->headers->userAgent = $config->userAgent;
 $result->headers->authorizationToken = $config->authorizationToken;
 
 return $result;
}
function createHeaderArray(GitHubRequestHeaders $headers): array {
 $result = array();
 
 array_push($result, "Accept: vnd.github.v3+json");
 
 if ( !empty($headers->userAgent) ) {
  array_push($result, "User-Agent: " . $headers->userAgent);
 }
 
 if ( !empty($headers->authorizationToken) ) {
  array_push($result, "Authorization: token " . $headers->authorizationToken);
 }
 
 if ( !empty($headers->etag) ) {
  array_push($result, "If-None-Match: " . $headers->etag);
 }
 
 return $result;
}
function performGet(GitHubGetRequest $request): GitHubGetResponse {
 $requestHeaderArray = createHeaderArray($request->headers);
 
 $handle = curl_init();
 curl_setopt($handle, CURLOPT_URL, $request->url);
 curl_setopt($handle, CURLOPT_FOLLOWLOCATION, true);
 curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
 curl_setopt($handle, CURLOPT_HTTPHEADER, $requestHeaderArray);
 
 $responseHeaderArray = array();
 curl_setopt(
  $handle,
  CURLOPT_HEADERFUNCTION,
  function($innerHandle, $currHeader) use (&$responseHeaderArray) {
   $headerPair = explode(":", $currHeader, 2);
   if ( count($headerPair) == 2 ) {
    $responseHeaderArray[strtolower(trim($headerPair[0]))][] = trim($headerPair[1]);
   }
   
   $innerResult = strlen($currHeader);
   return $innerResult;
  }
 );
 
 $responseBody = curl_exec($handle);
 
 $result = new GitHubGetResponse();
 if ( curl_errno($handle) == 0 ) {
  $result->code = curl_getinfo($handle, CURLINFO_RESPONSE_CODE);
  $result->headers = $responseHeaderArray;
  $result->body = $responseBody;
 } else {
  $result->curlError = new GitHubCurlError();
  $result->curlError->errorNum = curl_errno($handle);
  $result->curlError->errorMessage = curl_error($handle);
 }
 
 return $result;
}
function createResponseHeaders(GitHubGetResponse $response): GitHubResponseHeaders {
 $result = new GitHubResponseHeaders();
 
 if ( !empty($response->headers["etag"]) ) {
  $result->etag = $response->headers["etag"][0];
 }
 
 if ( !empty($response->headers["x-ratelimit-remaining"]) ) {
  $result->rateLimitRemaining = $response->headers["x-ratelimit-remaining"][0];
 }
 
 if ( !empty($response->headers["x-ratelimit-used"]) ) {
  $result->rateLimitUsed = $response->headers["x-ratelimit-used"][0];
 }
 
 if ( !empty($response->headers["x-ratelimit-reset"]) ) {
  $result->rateLimitResetUtcEpoch = $response->headers["x-ratelimit-reset"][0];
 }
 
 return $result;
}
?>

GitHubRepoMilestonesApi.php:

<?php
namespace Riley\GitHubApi;
include_once "GitHubApiUtils.php";
class RepoMilestoneDetails {
 public int $number;
 public string $milestoneUrl;
 public string $title;
 public string $description;
 public string $state;
 public ?string $dueDatetimeString = null;
 public ?string $closedDatetimeString = null;
 public int $openIssueCount;
 public int $closedIssueCount;
}
class CachedRepoMilestoneList {
 public string $etag;
 public array $milestoneList;
}
class RepoMilestoneListRequest {
 public string $username;
 public string $repoName;
 public ?string $state = null;
 public ?string $sort = null;
 public ?string $direction = null;
 public ?int $perPage = null;
 public ?int $page = null;
}
class RepoMilestoneListResponse {
 public GitHubCurlError $curlError;
 public GitHubResponseHeaders $headers;
 public array $milestoneList;
}
class RepoMilestoneClientService {
 
 private const MILESTONE_LIST_MAP_CACHE_FILENAME = "../../../cache/github/repo-milestone-list-map.json";
 
 private GitHubApiConfig $config;
 
 public function __construct(GitHubApiConfig $config) {
  $this->config = $config;
 }
 
 public function getRepoMilestoneList(RepoMilestoneListRequest $request): RepoMilestoneListResponse {
  $url = $this->resolveRepoMilestoneListUrl($request);
  
  $milestoneListMapCache = $this->loadRepoMilestoneListMapCache();
  $cachedMilestoneList = null;
  if ( isset($milestoneListMapCache[$url]) ) {
   $cachedMilestoneList = $milestoneListMapCache[$url];
  }
  
  $response = $this->performRepoMilestoneListRequest($url, $cachedMilestoneList);
  
  $result = new RepoMilestoneListResponse();
  if ( isset($response->curlError) ) {
   $result->curlError = $response->curlError;
   
  } else if ( $response->code == 304 ) {
   $result->headers = createResponseHeaders($response);
   $result->milestoneList = $cachedMilestoneList->milestoneList;
   
  } else if ( $response->code == 200 ) {
   $result->headers = createResponseHeaders($response);
   $result->milestoneList = $this->createMilestoneList($response->body);
   
   if ( $cachedMilestoneList == null ) {
    if ( !empty($result->headers->etag) ) {
     $cachedMilestoneList = new CachedRepoMilestoneList();
     $cachedMilestoneList->etag = $result->headers->etag;
     $cachedMilestoneList->milestoneList = $result->milestoneList;
     
     $milestoneListMapCache[$url] = $cachedMilestoneList;
     
     $this->saveRepoMilestoneListMapCache($milestoneListMapCache);
    }
    
   } else {
    $cachedMilestoneList->milestoneList = $result->milestoneList;
    
    $milestoneListMapCache[$url] = $cachedMilestoneList;
    
    $this->saveRepoMilestoneListMapCache($milestoneListMapCache);
   }
  }
  
  return $result;
 }
 
 private function resolveRepoMilestoneListUrl(RepoMilestoneListRequest $request): string {
  $result = "https://api.github.com/repos/" . $request->username . "/" . $request->repoName . "/" . "milestones";
  
  $queryStringArray = array(
   "state" => $request->state,
   "sort" => $request->sort,
   "direction" => $request->direction,
   "per_page" => $request->perPage,
   "page" => $request->page,
  );
  
  if ( count($queryStringArray) > 0 ) {
   $result = $result . "?" . http_build_query($queryStringArray);
  }
  
  return $result;
 }
 
 private function loadRepoMilestoneListMapCache(): array {
  $result = array();
  
  if ( file_exists(self::MILESTONE_LIST_MAP_CACHE_FILENAME) ) {
   $fileHandle = fopen(self::MILESTONE_LIST_MAP_CACHE_FILENAME, "r");
   
   if ( flock($fileHandle, LOCK_SH) ) {
    $contents = fread($fileHandle, filesize(self::MILESTONE_LIST_MAP_CACHE_FILENAME));
    flock($fileHandle, LOCK_UN);
    
    $jsonCache = json_decode($contents, true);
    
    foreach( $jsonCache as $url => $jsonMilestoneList ) {
     $currCachedMilestoneList = new CachedRepoMilestoneList();
     $currCachedMilestoneList->etag = $jsonMilestoneList["etag"];
     $currCachedMilestoneList->milestoneList = array();
     
     foreach( $jsonMilestoneList["milestoneList"] as $jsonMilestone ) {
      $currMilestoneDetails = new RepoMilestoneDetails();
      
      foreach( $jsonMilestone as $propertyName => $propertyValue ) {
       $currMilestoneDetails->{$propertyName} = $propertyValue;
      }
      
      array_push($currCachedMilestoneList->milestoneList, $currMilestoneDetails);
     }
     
     $result[$url] = $currCachedMilestoneList;
    }
   }
   
   fclose($fileHandle);
  }
  
  return $result;
 }
 
 private function saveRepoMilestoneListMapCache(array &$milestoneListMapCache) {
  $fileHandle = fopen(self::MILESTONE_LIST_MAP_CACHE_FILENAME, "w");
  
  if ( flock($fileHandle, LOCK_EX) ) {
   fwrite($fileHandle, json_encode($milestoneListMapCache, JSON_PRETTY_PRINT));
   fflush($fileHandle);
   
   flock($fileHandle, LOCK_UN);
  }
  
  fclose($fileHandle);
  
  clearstatcache(true, self::MILESTONE_LIST_MAP_CACHE_FILENAME);
 }
 
 private function performRepoMilestoneListRequest(string $url, ?CachedRepoMilestoneList &$cachedMilestoneList): GitHubGetResponse {
  $request = createGetRequest($url, $this->config);
  if ( $cachedMilestoneList != null ) {
   $request->headers->etag = $cachedMilestoneList->etag;
  }
  
  $result = performGet($request);
  return $result;
 }
 
 private function createMilestoneList(string $responseBody): array {
  $responseAsArray = json_decode($responseBody, true);
  
  $result = array();
  foreach( $responseAsArray as $currResponseItem ) {
   $currMilestoneDetails = new RepoMilestoneDetails();
   $currMilestoneDetails->number = $currResponseItem["number"];
   $currMilestoneDetails->milestoneUrl = $currResponseItem["html_url"];
   $currMilestoneDetails->title = $currResponseItem["title"];
   $currMilestoneDetails->description = $currResponseItem["description"];
   $currMilestoneDetails->state = $currResponseItem["state"];
   $currMilestoneDetails->dueDatetimeString = $currResponseItem["due_on"];
   $currMilestoneDetails->closedDatetimeString = $currResponseItem["closed_at"];
   $currMilestoneDetails->openIssueCount = $currResponseItem["open_issues"];
   $currMilestoneDetails->closedIssueCount = $currResponseItem["closed_issues"];
   
   array_push($result, $currMilestoneDetails);
  }
  
  return $result;
 }
 
}
?>

get_website_updates.php:

<?php
namespace Riley\GitHubApi;
include "../../../source/php/RileySiteUtils.php";
include "../../../source/php/RileyGitHubApiConfig.php";
include "../../../source/php/GitHubRepoMilestonesApi.php";
function convertMilestoneDetailsToSummary(RepoMilestoneDetails &$milestoneDetails): array {
 $titlePair = explode(" - ", $milestoneDetails->title, 2);
 $titleText = $titlePair[count($titlePair) - 1];
 
 $dueDateTime = !isset($milestoneDetails->dueDatetimeString) ? null : new \DateTime($milestoneDetails->dueDatetimeString);
 $closedDateTime = !isset($milestoneDetails->closedDatetimeString) ? null : new \DateTime($milestoneDetails->closedDatetimeString);
 
 $result = array(
  "title" => $titleText,
  "description" => $milestoneDetails->description,
  "dueDateTime" => \Riley\toIsoDateTimeString($dueDateTime),
  "closedDateTime" => \Riley\toIsoDateTimeString($closedDateTime),
  "openIssueCount" => $milestoneDetails->openIssueCount,
  "closedIssueCount" => $milestoneDetails->closedIssueCount,
 );
 return $result;
}
function convertMilestoneDetailsListToSummaryList(array &$milestoneDetailsList): array {
 $result = array();
 
 foreach( $milestoneDetailsList as $currMilestoneDetails ) {
  array_push($result, convertMilestoneDetailsToSummary($currMilestoneDetails));
 }
 
 return $result;
}
$config = RileyGitHubApiConfig::getWebsiteUpdatesConfig();
$milestoneService = new RepoMilestoneClientService($config);
$request = new RepoMilestoneListRequest();
$request->username = \RILEY\GITHUB_USER;
$request->repoName = \RILEY\GITHUB_SITE_REPO;
$request->state = "open";
$request->sort = "due_on";
$request->direction = "asc";
$request->per_page = 3;
$upcomingMilestoneListResponse = $milestoneService->getRepoMilestoneList($request);
$request->state = "closed";
$request->direction = "desc";
$recentMilestoneListResponse = $milestoneService->getRepoMilestoneList($request);
if (
 !isset($upcomingMilestoneListResponse->milestoneList) ||
 !isset($recentMilestoneListResponse->milestoneList)
) {
 http_response_code(404);
} else {
 $websiteUpdatesResponse = array(
  "upcomingMilestoneList" => convertMilestoneDetailsListToSummaryList($upcomingMilestoneListResponse->milestoneList),
  "recentMilestoneList" => convertMilestoneDetailsListToSummaryList($recentMilestoneListResponse->milestoneList),
 );
 
 header("Content-Type: application/json");
 echo json_encode($websiteUpdatesResponse, JSON_PRETTY_PRINT);
}
?>