Build a blog application on Google App Engine: BlogPost module (part 4)
This is the fourth part of a multipart tutorial on how to build a blog application in Node.js using the Google Datastore and deploy it to Google App Engine. If you missed the beginning, jump to the first part where I explain how to set up the project and where you’ll find the links to the other parts.
In this post, we are going to create our hand dirty with the first module: the BlogPost module. If you do remember, in part 2 of this tutorial, I mentioned a Blog module, not BlogPost… This is because our Blog module will contain 2 sub-modules: BlogPost and Comment, and we are just about to build the first one. Let’s go for it!
Add a “blog-post” folder inside the “modules/blog” folder and create its entry file, “index.ts” with the following content:
// modules/blog/blog-post/index.tsimport { Context, Modules } from "../models";export * from "./models";export interface BlogPostModule {}export default (
context: Context,
modules: Modules
): BlogPostModule => {
return {};
};
We can see that the module — like all our layers so far — requires 2 Input properties (context and modules) and returns the BlogpostModule interface. We haven’t created yet the Context and Modules Type so let’s do that right now. Create a “models.ts” file at the root of the Blog module and add the following:
// modules/blog/models.tsimport { Gstore } from "gstore-node";
import { Logger } from "winston";
import { ImagesModule } from '../images/index';
import { UtilsModule } from '../utils/index';
import { BlogPostModule } from "./blog-post";export type Context = {
gstore: Gstore,
logger: Logger
};export type Modules = {
blogPost?: BlogPostModule;
images?: ImagesModule;
utils?: UtilsModule;
};
You might ask: “Why don’t we use the Context Type that we already declared in our root “models.ts” file?”. Well, this would tie our Blog module to the application it is injected into (we would have to import the Context Type from outside the module root with a relative path like this: “import ../../models.ts”). This makes the module less portable and we wouldn’t be able to expose it as a separate package (like npm) if we would need it.
Let’s now creates the Typescript BlogPost Type that will define our BlogPost Entity Kind in Google Datastore. Create a “models.ts” file inside the “modules/blog/blog-post” folder and add the following:
// modules/blog/blog-post/models.tsexport type BlogPostType = {
title?: string,
createdOn?: Date,
modifiedOn?: Date,
content?: string,
contentToHtml?: string,
excerpt?: string,
posterUri?: string,
cloudStorageObject?: string
};
Not much to explain here. We will save the content of the blog post in markdown format. We will then convert it at runtime into HTML and store it in the contentToHtml property.
Let’s now instantiate our module. Open the “modules/blog/index.ts” file and make the following modifications:
// modules/blog/index.tsimport initBlogPost, { BlogPostModule } from "./blog-post";
import { Context, Modules } from "./models";export interface BlogModule {
blogPost: BlogPostModule;
}export default (context: Context, modules: Modules): BlogModule => {
const blogPost = initBlogPost(context, {}); return {
blogPost,
};
};
The last thing to do is to provide the context and modules inputs when we instantiate the Blog module. Open the “modules.ts” file at the root of the server src folder and make the following modification:
// modules.ts...export default (context: Context): AppModules => {
const utils = initUtilsModule();
const images = initImagesModule();
const blog = initBlogModule(context, { utils, images }); // Update
...
Great. We have initialised our Blog module providing its dependencies (Utils module and Image module). The Blog module then initialises the BlogPost sub-module and returns its instance as part of the BlogModule Type.
BlogPost Routes Handlers
Let’s now create the route handlers for our BlogPosts. These handlers will be the methods called by the Express router when a URL is matched. For our BlogPost we will need to:
- List posts (
GET /blogroute) - View post detail (
GET /blog/<post-id>route)
Create a “blog-post.routes-handlers.ts” file inside the “blog-post” sub-module and add the following:
// modules/blog/blog-post/blog-post.routes-handlers.tsimport { Request, Response } from "express";
import { Context, Modules } from "../models";export interface BlogPostRoutesHandlers {
listPosts(req: Request, res: Response): any;
detailPost(req: Request, res: Response): any;
}export default (
{ gstore, logger }: Context,
modules: Modules
): BlogPostRoutesHandlers => {
return {};
};
We have first declared the interface of the routes handlers layer and then defined it as output. It is a good habit to declare the interface first like we just did, this way we don’t need to add any types to our methods signature later on. We have then declared the inputs (context, and modules) and deconstructed the gstore and logger instances from the Context object.
Let’s now add the 2 methods from our interface.
// modules/blog/blog-post/blog-post.routes-handlers.ts...export default (
{ gstore, logger }: Context,
modules: Modules
): BlogPostRoutesHandlers => {
return {
async listPosts(_, res) {
const template = "blog/index"; res.render(template, {
blogPosts: [],
pageId: "home"
});
},
async detailPost(req, res) {
const template = "blog/detail"; return res.render(template, {
post: {},
pageId: "blogpost-view",
});
}
};
};
We have created our 2 routes handlers and declared them as async, even though — for now — there is no asynchronous code in them. Let’s instantiate this layer and export it. Open the “index.ts” from our blog-post sub-module.
// modules/blog/blog-post/index.tsimport initRoutes, {
BlogPostRoutesHandlers
} from "./blog-post.routes-handlers";
import { Context, Modules } from "../models";export * from "./models";export interface BlogPostModule {
routesHandlers: BlogPostRoutesHandlers;
}export default (
context: Context,
modules: Modules
): BlogPostModule => {
return {
routesHandlers: initRoutes(context, {})
};
};
Now that we have our route handlers created, we need a router to attach them. Create a “blog.routes.ts” file at the root of the Blog module.
// modules/blog/blog.routes.tsimport express from "express";
import { Context, Modules } from "./models";export default (context: Context, { blogPost }: Modules) => {
// WEB
const webRouter = express.Router(); webRouter.get("/", blogPost.routesHandlers.listPosts);
webRouter.get("/:id", blogPost.routesHandlers.detailPost); return {
webRouter
};
};
We created an Express router (we called it webRouter because later we will create another router for an API that we will need) and mapped our handlers to the “/” and “/:id” paths. This makes this module very portable. It is the parent router that will decide from which path to make it accessible. Let’s now instantiate the routes: open the “index.ts” file from our Blog module and make the following modifications:
// modules/blog/index.tsimport { Router } from "express"; // Add this line
import initRoutes from "./blog.routes"; // Add this line
import initBlogPost, { BlogPostModule } from "./blog-post";
import { Context, Modules } from "./models";export interface BlogModule {
webRouter: Router; // Add
blogPost: BlogPostModule;
}export default (context: Context, modules: Modules): BlogModule => {
const blogPost = initBlogPost(context, {});
const { webRouter } = initRoutes(context, { blogPost }); // Addreturn {
webRouter, // Add this line
blogPost
};
};
Finally, we need to attach this router to our main application router. Open the “routes.ts” file at the root and make the following modification:
// routes.ts...export default (
{ logger, config }: Context,
{
app,
modules: { blog, admin }
}: { app: Express, modules: AppModules }
) => {
/**
* Web Routes
*/
app.use("/blog", blog.webRouter); // update this line
...
};
Great! If you refresh your browser window, you should see the home page of our blog at the /blog URL. If you navigate to /blog/123 you should see the skeleton of our detail view. Great!
Let’s pause here for a second and do a quick recap of what we have just done.
- Main application router → we have declared a
/blogpath and let the Blog module router handle it. - Blog module router → we’ve exported a web router that handles 2 routes:
/to list our posts and/:idto show the detail of a post. - We’ve created two routes handlers in our BlogPost sub-module to handle the routes from our Blog module.
By keeping the separation of concerns to each module and layer we are building an application that can scale easily in terms of code and at the same time making our modules very portables. We are currently building a monolithic application as it would be overkill at this stage to create microservices. But if the time comes and we need to expose our Images our Blog module as an independent service, it wouldn’t be too much work to do.
BlogPost Database
Let’s create now the Database access layer for our BlogPost. At last some interaction with the Datastore!
Create a “blog-post.db.ts” file in the BlogPost sub-module folder and add the following into it:
// modules/blog/blog-post/blog-post.db.tsimport {
Entity,
QueryListOptions,
QueryResult,
DeleteResult
} from "gstore-node";
import { Context, Modules } from "../models";
import { BlogPostType } from "./models";export interface BlogPostDB {
getPosts(
options?: QueryListOptions
): Promise<QueryResult<BlogPostType>>;
getPost(
id: number,
dataloader: any,
format?: string
): Promise<Entity<BlogPostType> | BlogPostType>;
createPost(
data: BlogPostType,
dataloader: any
): Promise<Entity<BlogPostType>>;
updatePost(
id: number,
data: any,
dataloader: any,
replace: boolean
): Promise<Entity<BlogPostType>>;
deletePost(id: number): Promise<DeleteResult>;
}export default (
context: Context,
{ images, utils }: Modules
): BlogPostDB => {
const { gstore } = context; /**
* DB API
*/
return {
// TODO
};
};
The code is quite self-explanatory but let’s go quickly over its content. First, we imported the Types from gstore-node and our “models.ts” files. We then declared the interface of this layer. Nothing fancy, normal CRUD methods.
You might have noticed that for the getPost() method the return Type is either a Promise of a gstore Entity instance or a Promise of a BlogPostType. This is because, according to the format argument passed, we will have 2 distinct return types.
The first thing to do in gstore-node to interact with an Entity Kind on the Datastore is to declare a Schema. Let’s do that and create the BlogPost schema:
// modules/blog/blog-post/blog-post.db.ts...export default (
context: Context,
modules: Modules
): BlogPostDB => {
const { gstore } = context;
const { Schema } = gstore; /**
* Schema for the BlogPost entity Kind
*/
const schema = new Schema<BlogPostType>({
title: { type: String },
createdOn: {
type: Date,
default: gstore.defaultValues.NOW,
read: false,
write: false
},
modifiedOn: { type: Date, default: gstore.defaultValues.NOW },
content: { type: String, excludeFromIndexes: true },
excerpt: { type: String, excludeFromIndexes: true },
posterUri: { type: String },
cloudStorageObject: { type: String }
}); const ancestor = ['Blog', 'default']; /**
* Configuration for our Model.list() query shortcut
*/
schema.queries('list', {
order: { property: 'modifiedOn', descending: true },
ancestors: ancestor,
}); ...
We created a gstore schema providing a Typescript type (Schema<BlogPostType>). We are defining the type twice indeed (once in Typescript and once in the gstore schema), but each one has a different purpose. Typescript will validate the types during development and compile time, letting us know instantly if we are breaking a contract. But what prevents us from inserting invalid data into the Datastore are the gstore types. Refer to the gstore-node documentation for all the information about schema declaration and configuration.
With the above schema declared, no other property can be added to a BlogPost Entity Kind in the Datastore. Sweet!
I’ll explain in a moment what the ancestor variable is. Right below, we configure the gstore Model.list() query shortcut that will allow us to easily query our BlogPosts entities and order them by themodifiedOn property.
We now need to create a gstore model for our schema. Add the following right below the schema.queries('list') call:
// modules/blog/blog-post/blog-post.db.ts.../**
* Create a "BlogPost" Entity Kind Model
*/
const BlogPost = gstore.model("BlogPost", schema);...
Great, now that we have our model we can build our BlogPostDB interface.
// modules/blog/blog-post/blog-post.db.ts...const BlogPost = gstore.model("BlogPost", schema);/**
* DB API
*/
return {
getPosts: BlogPost.list.bind(BlogPost),
getPost(id, dataloader, format = "JSON") {
return BlogPost.get(id, ancestor, null, null, {
dataloader
}).then(entity => {
if (format === "JSON") {
// Transform the gstore "Entity" instance
// to a plain object (adding an "id" prop to it)
return entity.plain();
}
return entity;
});
},
createPost(data, dataloader) {
const post = new BlogPost(data, null, ancestor); // We add the DataLoader instance to our entity context
// so it is available in our "pre" Hooks
post.context.dataloader = dataloader;
return post.save();
},
updatePost(id, data, dataloader, replace) {
return BlogPost.update(id, data, ancestor, null, null, {
dataloader,
replace
});
},
deletePost(id) {
return BlogPost.delete(id, ancestor);
},
};
Let’s see what happens in each method of our CRUD interface.
getPosts()
This method is simply a proxy to the gstore Model.list() shortcut query. Refer to the documentation to see what arguments it accepts.
getPost()
This method accepts 3 arguments.
- The id of the post to retrieve.
- A dataloader instance.
Dataloader is an utility that optimises entities fetching from a database by gathering multiple db access call ( which occur within a single tick of the event loop) into a single one (perfect for the Google Datastore as we can get multiple entities by Key in one single call) and by caching the result of those requests. It is not meant to be used as a LRU cache layer but to optimise data fetching happening in a single Http Request.
For a detail explanation, have a look here or the official repo.
- The format of the response. It can either be a plain JSON object of the data stored in the Datastore, or agstore entity instance.
createPost()
Not much to say here. We provide the entity data to be saved and optionally a dataloader instance.
updatePost()
Similar to the createPost() but obviously here we need to provide an id to update the post. We can also specify if we want to completely replace the data in the Datastore or if we want to merge the data provided with the data currently in the Datastore.
deletePost()
We are simply calling the corresponding gstore Model.delete() method.
Enforce strong consistency
You have noticed that we are using in multiple places the variable ancestor. We will save all our BlogPosts under the Datastore Ancestor path: ["Blog", “default"], which corresponds to a parent Blog entity kind and we gave it a “default” name. This entity does not need to exist in the Datastore, we are just saving all our posts as one of its children and thus in the same entity group. By doing that, we are getting strong consistency on the Datastore. You can read more about it on Google Cloud.
Let’s now initialise this layer. For that, open the “index.ts” of our blog-post folder.
// modules/blog/blog-post/index.tsimport initDB from "./blog-post.db"; // Add this line
import initRoutes, {
BlogPostRoutesHandlers
} from "./blog-post.routes-handlers";...export default (
context: Context,
modules: Modules
): BlogPostModule => {
const blogPostDB = initDB(context, modules); // Add this line return {
routesHandlers: initRoutes(context, {})
};
};
BlogPost domain
We have created our routes handlers, we have a database layer to interact with the Datastore. We now need a domain layer to connect one to the other. Let’s go for it!
Create a “blog-post.domain.ts” file in the blog-post folder and add the following:
// modules/blog/blog-post/blog-post.domain.tsimport marked from "marked";
import Boom from "boom";
import {
Entity,
QueryListOptions,
QueryResult,
DeleteResult
} from "gstore-node";
import { Context, Modules } from "../models";
import { BlogPostType } from "./models";export interface BlogPostDomain {
getPosts(
options?: QueryListOptions
): Promise<QueryResult<BlogPostType>>;
getPost(
id: number | string,
dataLoader: any
): Promise<Entity<BlogPostType>>;
createPost(
data: BlogPostType,
dataLoader: any
): Promise<Entity<BlogPostType>>;
updatePost(
id: string | number,
data: BlogPostType,
dataloader: any,
overwrite?: boolean
): Promise<Entity<BlogPostType>>;
deletePost(id: string | number): Promise<DeleteResult>;
}export default (
context: Context,
{ blogPostDB }: Modules
): BlogPostDomain => {
return {};
};
Nothing fancy here, we declare the interface and return the layer initializer. We can see that we have a dependency on the blogPostDB layer from the Modules type. Let’s add it quickly.
// modules/blog/models.ts...import { BlogPostModule } from "./blog-post";
import { BlogPostDB } from "./blog-post/blog-post.db"; // Add thisexport type Context = {
gstore: Gstore;
logger: Logger;
};export type Modules = {
blogPost?: BlogPostModule;
blogPostDB?: BlogPostDB; // Add this line
images?: ImagesModule;
utils?: UtilsModule;
};
Great, we can now finish the BlogPostDomain interface
// modules/blog/blog-post/blog-post.domain.ts...export default (
context: Context,
{ blogPostDB }: Modules
): BlogPostDomain => {
return {
/**
* Get a list of BlogPosts
*/
getPosts(options = {}) {
return blogPostDB.getPosts(options);
},
/**
* Get a BlogPost
* @param {*} id Id of the BlogPost to fetch
* @param {*} dataloader optional. A Dataloader instance
*/
async getPost(id, dataloader) {
id = +id;
if (isNaN(id)) {
throw new Error("BlogPost id must be an integer");
} let post: Entity<BlogPostType>;
try {
post = <Entity<BlogPostType>>(
await blogPostDB.getPost(id, dataloader)
);
} catch (err) {
throw err;
} if (post && post.content) {
// Convert markdown to Html
post.contentToHtml = marked(post.content);
} return post;
},
/**
* Create a BlogPost
* @param {*} data BlogPost entity data
* @param {*} dataloader optional Dataloader instance
*/
createPost(data: BlogPostType, dataloader: any) {
return blogPostDB.createPost(data, dataloader);
},
/**
* Update a BlogPost entity in the Datastore
* @param {number} id Id of the BlogPost to update
* @param {*} data BlogPost entity data
* @param {Dataloader} dataloader optional Dataloader instance
* @param {boolean} overwrite overwrite the entity in Datastore
*/
updatePost(id, data, dataloader, overwrite = false) {
id = +id;
if (isNaN(id)) {
throw new Boom("BlogPost id must be an integer", {
statusCode: 400
});
} return blogPostDB.updatePost(id, data, dataloader, overwrite);
},
/**
* Delete a BlogPost entity in the Datastore
* @param {number} id Id of the BlogPost to delete
*/
deletePost(id) {
return blogPostDB.deletePost(+id);
}
};
};
I will not go into the code as I think it is self-explanatory. I will only put emphasis on the importance of this layer. This is where the logic, unique to our application, resides. The routes handler are just connectors from the Express router to this layer. The DB layer is just a facade for this layer to the Datastore. Here is where all the magic happens :)
Although we only need for now to list posts and view a post detail, we already declared the other operations (create, update, delete) that we will need when building the Admin module. Let’s now connect the route handlers to our newly defined domain methods.
// modules/blog/blog-post/blog-post.routes-handlers.tsimport { Request, Response } from 'express';
import { Entity, QueryResult, DeleteResult } from 'gstore-node';
import { BlogPostType } from './models';
import { Context, Modules } from '../models';...export default (
{ gstore }: Context,
{ blogPostDomain }: Modules
): BlogPostRoutesHandlers => {
return {
async listPosts(_, res) {
const template = "blog/index";
let posts: QueryResult<BlogPostType>; try {
posts = await blogPostDomain.getPosts();
} catch (error) {
return res.render(template, {
blogPosts: [],
error
});
} res.render(template, {
blogPosts: posts.entities,
pageId: "home"
});
},
async detailPost(req, res) {
/**
* Create Dataloader instance, unique to this Http Request
*/
const dataloader = gstore.createDataLoader();
const template = "blog/detail"; let blogPost: Entity<BlogPostType>;
try {
blogPost = await blogPostDomain.getPost(
req.params.id,
dataloader
);
} catch (error) {
if (error.code === "ERR_ENTITY_NOT_FOUND") {
return res.redirect("/404");
}
return res.render(template, { post: null, error });
} return res.render(template, {
pageId: "blogpost-view",
blogPost
});
}
};
};
Let’s initialize this layer and provide it in our Modules type.
// modules/blog/blog-post/index.tsimport initDB from "./blog-post.db";
import initRoutes, {
BlogPostRoutesHandlers
} from "./blog-post.routes-handlers";
import initDomain, { BlogPostDomain } from "./blog-post.domain";...export interface BlogPostModule {
blogPostDomain: BlogPostDomain; // Add this line
routesHandlers: BlogPostRoutesHandlers;
}export default (
context: Context,
modules: Modules
): BlogPostModule => {
const blogPostDB = initDB(context, modules);
const blogPostDomain = initDomain(context, { blogPostDB }); // Add return {
blogPostDomain, // Add this line
routesHandlers: initRoutes(context, { blogPostDomain })
};
};
And update our Type…
// modules/blog/models.ts...import { BlogPostDB } from './blog-post/blog-post.db';
import { BlogPostDomain } from './blog-post/blog-post.domain';// Add...export type Modules = {
blogPost?: BlogPostModule;
blogPostDB?: BlogPostDB;
blogPostDomain?: BlogPostDomain; // Add this line
images?: ImagesModule;
utils?: UtilsModule;
};
And with this, we are done (for now) with the BlogPost module. It has been a long post as I tried to explain all the moving parts. I hope I didn’t get you lost along the way. As always, please reach out in the comments below if something is not clear.
For the next modules that we will build, I won’t go into so many details as we will follow the exact same patterns seen here.
It is time to start generating some content for our Blog! This is what we will do in the next part of this tutorial: build an Admin module to manage our posts.
Thanks for reading!
