A Newbie Gopher’s Server Developing Experience

Previous Story: Getting Used to Go

Some Coin’s Price Prediction Model

Introduction

After playing a bit with Go, I decided to do something bigger; to make web service of something. Since it is said Go is good for server development, I think it would be great to implement some web server that manages data and serve them.

At that time, I did some activities in my university’s alumni group. The group was about financial investments, so I had chances to have some coffee chats with couple fund managers. One assets that we talked a lot were cryptocurrencies

One fund manager told me that it would be fun to build some web service out of his logic and my SW skills. Here’s what we have gone through;

Idea

I’m not going to be explaining some wacky mechanism of cryptocurrency internals or fundamentals. Over simplified conclusion of our logic is this;

Total burned USD might be related to cryptocurrency’s prices

What is total burned USD? Ever heard of stock buyback? Same thing for crypto is burn. Some cryptos are issued by certain subject and the subjects buyback issued crypto with USD or other traditional currencies.

We thought it would be fun to track long term data of price and burn. Furthermore it might be a great research if we make relation equations out of it, and digitize how it matches.

In order to do this, basic steps are same as follows;

1. Periodically fetch price and burn amount
2. Stack it someware to make historical data
3. Calculate prediction from historical data

Design

After discussing this idea with the fund manager, I started to draw some simple overview of system.

Simple overview of architecture and tasks

Client would be simple HTML/CSS/JS to provide view to user. Client will request data to server, and server will provide data. Server has various role of managing, caching, fetching, calculating data.

Here’s 2 major task that happens

Purple Task: Data Fetching and Processing

1. Server periodically fires purple task with scheduler(once a week)
2. Fetch data from external sources
3. Process data: convert to db data, calculate price prediction model
4. Cache data: since the data is relatively static(it updates once a week), cache could be a good choice. Refer to this package that I implemented
5. Save it to DB: even if we have cache, the true data should be stored to persistent storage. DB is the choice

Yellow Task: User Lands to Web Page

1. User visit web page and JavaScript logic will call server to get data.
2. Server checks cache first. If hit, move to 4th step
3. If cache doesn’t hit, server will retrieve data from DB, save it to cache
4. Response with data

Some Learnings

Since basic idea of this projects are not from me, I thought it would be better not to disclosure every details of it, so I am not giving repository of this project.

Instead, I’m introducing some key learnings of this projects

• Basic HTTP Server Design Patterns
• Use of Cache
• Use of Go Package Publishing

Basic HTTP Server Design Patterns

Internal folder of satellity repository

While I am developing server, I heavily referenced the structure of satellity. Key folders to watch are controller, model which are basically representing MVC pattern.

Go files in controller acts as controller for http server. If request comes, it parses, checks error, and calls model function according to request method. Go files in model acts as source of data structure and provide functions to modify or retrieve it.

I followed the same pattern for my project.

Use of Cache

Cache

Next key learning is use of cache. It was closely related to our business logic that the data updates once a week. In that case, cache is pretty good because user will request same data for entire week.

Cache is temporary storage that smaller but faster than database. For handling frequent and static request, using cache will enhance performance. Cache have some fallbacks with data integrity when data changes frequently but this is not the case for us.

I implemented simple key-value store Go package called CStore(abbreviation of Cache Store) and used it in server.

Use of Go Package Publishing

Published Packages: CStorage(left) and PRegression(right)

Last key point of learn is Go package publishing. Since I used Node.js for couple years, I had similar experience of publishing some useful functions to npm. Same here, I made 2 packaged published during this project.

First one is Cache Storage (https://pkg.go.dev/github.com/cocm1324/cstorage). Cache would be useful to other server kind of projects.

Second one is Polynomial Regression (https://pkg.go.dev/github.com/cocm1324/pregression). Polynomial regression served as great utility for data processing in this project but I thought this package can be used in another financial or data processing project in the future.

Conclusion

As a newbie Gopher, I learned a lot from implementing server with Go.

I discussed idea with fund manager, draw architecture from the idea, implemented it. Now, about 100 alumni in my group visits the website regularly.

I learned basic pattern of Go server, advantage of using cache with static data, and how to publish package for future use.

Go story will return;) Thanks for watching.

Next Story: Implementing Cache With Go