DATA STRUCTURES: JSON

Working with JSON in Go

JSON is a text-based data exchange format primarily used between browsers and servers. In this article, we are going to look at JSON encoding and decoding APIs provided by the Go.

Uday Hiwarale
Mar 1, 2020 · 22 min read
(source: unsplash.com)
{
"firstName": "John",
"lastName": "Doe",
"age": 21,
"heightInMeters": 1.75,
"isMale": true,
"profile": null,
"languages": [ "English", "French" ],
"grades": {
"math": "A",
"science": "A+"
}
}
(Google Chrome DevTools)
[
"John Doe",
21,
[ "English", "French" ],
{
"math": "A",
"science": "A+"
}
]

Encoding JSON

To encode a JSON from a suitable data structure, we use json.Marshal function provided by the json package. This function has the following syntax.

func Marshal(v interface{}) ([]byte, error)
(https://play.golang.org/p/GW8lhgat7HQ)
{"FirstName":"John","Email":"","Age":21,"HeightInMeters":1.75,"IsMale":true}
(https://play.golang.org/p/1lbJQRu7Up5)
{"Age":21,"FirstName":"John","HeightInMeters":1.75,"IsMale":true,"lastName":"Doe"}

Data Types Handling

As we have learned, JSON supports primarily 6 data types viz. string, number, boolean, null, array and object. This can be great news for a JavaScript developer because all data types are supposed in JavaScript, but in Go, we need to consider various data types while encoding.

  1. String: A string value is (raw or interpreted string literal) is sanitized and encoded as JSON string. Read more about the sanitization process from this documentation. The []byte value is encoded as a Base64 string.
  2. Boolean: A bool value is encoded as JSON boolean value.
  3. Null: A nil value (like of a pointer, interface or other data types) is encoded as JSON null value.
  4. Object: A map or a struct value is encoded as JSON object value.
  5. Array: An array or a slice value is encoded as JSON array value except for the slice of bytes ([]byte).

Abstract Data Types

In the previous examples, we have encoded values of concrete data types like int, string, bool etc. Let’s add more complex data values like struct, map and interface to an object and see how it encodes to the JSON.

(https://play.golang.org/p/2BfjZ2eEYT3)
{
"FirstName": "John",
"Age": 21,
"Profile": {
"Username": "johndoe91",
"Grades": {
"Math": "A",
"Science": "A+"
}
},
"Languages": [
"English",
"French"
]
}
(https://play.golang.org/p/crOcaf8NZyF)
{
"FirstName": "John",
"Age": 21,
"Username": "johndoe91",
"Grades": null,
"Languages": [
"English",
"French"
]
}
(https://play.golang.org/p/TjBOmLCPp9f)
{
"FirstName": "John",
"Age": 21,
"Primary": {
"Username": "johndoe91",
"Followers": 1976
},
"Secondary": null
}

Data Type Conversion

Sometimes, we do not want to encode a field value as it is but to provide a custom value for marshaling. This can be achieved by implementing json.Marshaler or encoding.TextMarshaler interface.

// from `encoding/json` package
type Marshaler interface {
MarshalJSON() ([]byte, error)
}
// from `encoding` package
type TextMarshaler interface {
MarshalText() (text []byte, err error)
}
(https://play.golang.org/p/xxg31Vd2i1d)
{
"FirstName": "John",
"Age": "{\"age\": 21}",
"Profile": {
"f_count": "1975"
}
}

Using Structure Tags

A struct field can hold additional metadata that can be used by other programs to process that field differently. This metadata is assigned to a field using a string literal (raw string [``] or interpreted string [“”]).

type Data struct {
FieldOne string `json:"fname" xml:"first-name" gorm:"size:255"`
FieldTwo string `json:"lname" xml:"last-name" gorm:"size:255"`
}
(https://play.golang.org/p/Tiy3M4VC9gX)
{
"fname": "John",
"-": 21,
"IsMale": "false",
"Profile": {
"uname": "johndoe91",
"followers": "1975"
}
}

Handling Maps

In the previous map example, we saw that a map with string or int can be encoded as JSON and all int keys are converted to JSON. However, map in Go can be more complex and its keys can be of a complex data type.

Decoding JSON

Decoding JSON is a little bit tricky because we need to coerce some text-based data into a complex data structure. To decode JSON into a valid data structure like map or struct, we first need to make sure if a JSON is valid.

func Valid(data []byte) bool
(https://play.golang.org/p/mPFKAiEjqxz)
func Unmarshal(data []byte, v interface{}) error
(https://play.golang.org/p/jYrIcYGlh8H)
Error: <nil>
main.Student{FirstName:"John", lastName:"", Email:"", Age:21, HeightInMeters:175}
Error: json: cannot unmarshal number 1.75 into Go struct field Student.HeightInMeters of type int

Handling Complex Data

If a JSON contains complex data like object or array, then a structure must declare the fields of appropriate types in order to unmarshal the JSON without an error.

https://play.golang.org/p/z1T8qF_HpMO
Error: <nil>
main.Student{
FirstName: "John",
lastName: "",
HeightInMeters: 1.75,
IsMale: true,
Languages: [2]string{
"English",
"Spanish"
},
Subjects: []string{
"Math",
"Science"
},
Grades: map[string]string{
"Math": "A",
"Science": "A+"
},
Profile: main.Profile{
Username: "johndoe91",
Followers: 1975
}
}
  1. If Unmarshal encounters an array type and array values in the JSON are more than the array can hold, then extra values are discarded. If array values in JSON are less than the length of the array, then the remaining array elements are set to their zero-values. The array type should be compatible with the values in the JSON.
  2. If Unmarshal encounters a slice type, then the slice in the struct is set to 0 length and elements from the JSON array are appended one at a time. If the JSON contains an empty array, then Unmarshal replaces the slice in the struct with an empty slice. The slice type should be compatible with the values in the JSON.
  3. If Unmarshal encounters a map type and the map’s value in the struct is nil, then a new map is created and object values in the JSON are appended. If the map value is non-nil, then the original value of the map is reused and new entries are appended. The map type should be compatible with the values in the JSON.
(https://play.golang.org/p/75wtjWQ71va)
Error: <nil>main.Student{FirstName:"John", lastName:"", HeightInMeters:1.75, IsMale:true, Languages:[2]string{"English", ""}, Subjects:[]string{"Math", "Science"}, Grades:map[string]string(nil), Profile:(*main.Profile)(0xc00000c080)}&main.Profile{Username:"johndoe91", Followers:1975}

Promoted Fields

If a structure contains anonymously nested struct field, the nested structure field will get promoted to the parent struct. Hence, the JSON must contain the field values on the parent object.

(https://play.golang.org/p/-7wNcmnY5w4)
Error: <nil>main.Student{FirstName:"John", lastName:"", HeightInMeters:1.75, IsMale:true, Profile:main.Profile{Username:"johndo
e91", Followers:1975}, Account:main.Account{IsMale:false, Email:""}}

Using Structure Tags

In the JSON encoding lesson, we learned that structure field tags can be very helpful to decide field names and omission criteria. We can also use the structure tags to interpolate JSON field names to struct field names.

(https://play.golang.org/p/VXkHsPCLJ9e)
Error: <nil>main.Student{FirstName:"John", LastName:"", HeightInMeters:1.75, IsMale:false, Languages:[]string(nil), Profile:main.Profile{Username:"johndoe91", Followers:0}}

Working with Maps

Since a JSON contains string keys and values of supported data types, a map of type map[string]interface{} is a suitable candidate for storing JSON data. We can pass a pointer to nil or non-nil pointer of the map to the Unmarshal function and all JSON field values will be populated inside the map.

(https://play.golang.org/p/yFDL7d-yVDO)
Error: <nil>
main.Student{"fname":"John", "height":1.75, "id":123, "languages":interface {}(nil), "male":true, "profile":map[string]interface {}{"f_count":1975, "uname":"johndoe91"}, "subjects":[]interface {}{"Math", "Science"}}
1: key `languages` of type `string` has value `<nil>` of type `<nil>`
2: key `subjects` of type `string` has value `[]interface {}{"Math", "Science"}` of type `[]interface {}`
3: key `profile` of type `string` has value `map[string]interface {}{"f_count":1975, "uname":"johndoe91"}` of type `map[string]interface {}`
4: key `id` of type `string` has value `123` of type `float64`
5: key `fname` of type `string` has value `"John"` of type `string`
6: key `height` of type `string` has value `1.75` of type `float64`
7: key `male` of type `string` has value `true` of type `bool`
  1. A JSON number value(int or float) is stored as float64.
  2. A JSON boolean value is stored as bool.
  3. A JSON null value is stored as nil value.
  4. A JSON array value is stored as a slice of type []interface{}.
  5. A JSON object value is stored as a map of type map[string]interface{}.
(https://play.golang.org/p/1-3VehLpjHZ)
Before: `type` of `john` is <nil> and its `value` is <nil>
Error: <nil>
After: `type` of `john` is map[string]interface {}
map[string]interface {}{"fname":"John", "height":1.75, "id":123, "languages":interface {}(nil), "male":true, "profile":map[string]interface {}{"f_count":1975, "uname":"johndoe91"}, "subjects":[]interface {}{"Math", "Science"}}
johnData := john.(map[string]interface{})

Using Unmarshaler and TextUnmarshaler

A struct field can take responsibility for unmarshaling the JSON data on its own. In such a case, the field value must implement the json.Unmarshaler interface which provides the declaration of UnmarshalJSON method.

type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
(https://play.golang.org/p/Uzl-TULVKkB)
container: map[string]interface {} / map[string]interface {}{"Username":"johndoe91", "f_count":1975}iuserName: string/"johndoe91"
ifollowers: float64/1975
userName: string/"johndoe91"
followers: float64/1975
Error: <nil>
main.Student{FirstName:"John", Profile:main.Profile{Username:"JOHNDOE91", Followers:"1.98k"}}

Encoder and Decoder

Go provides json/Encoder and json/Decoder structure types to encode JSON from a data stream and decode JSON to a data stream. This is helpful to process JSON as some data is available.

Encoder

The json/Encoder structure type lets you create a struct that holds a io.Writer object and provides Encode() method to encode JSON from an object and write to this io.Writer object.

func (enc *Encoder) Encode(v interface{}) error
func NewEncoder(w io.Writer) *Encoder
(https://play.golang.org/p/lBZeZOpbAGA)

Decoder

The json/Decoder structure type lets you create a struct that holds a io.Reader object and provides Decode() method to decode JSON from this io.Writer object and write to an object.

func (dec *Decoder) Decode(v interface{}) error
func NewDecoder(r io.Reader) *Decoder
(https://play.golang.org/p/SWgAD8BOVp2)

RunGo

A go-to guide for learning Go programming language

Uday Hiwarale

Written by

Software Engineer at kausa.ai / thatisuday.com ☯ github.com/thatisuday ☯ thatisuday@gmail.com

RunGo

RunGo

A place to find introductory Go programming language tutorials and learning resources. In this publication, we will learn Go in an incremental manner, starting from beginner lessons with mini examples to more advanced lessons.

Uday Hiwarale

Written by

Software Engineer at kausa.ai / thatisuday.com ☯ github.com/thatisuday ☯ thatisuday@gmail.com

RunGo

RunGo

A place to find introductory Go programming language tutorials and learning resources. In this publication, we will learn Go in an incremental manner, starting from beginner lessons with mini examples to more advanced lessons.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface.

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox.

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic.

Get the Medium app