Slice Tricks in Go

2 min readJul 2, 2019

I created the dataframe-go package to explore and manipulate mammoth amounts of data. So far, it’s served me well. Behind the scenes, it uses slices. I used various tricks to make the package as performant as possible. Hopefully, this article will be clearer than the official SliceTricks wiki page.

Since slices are ubiquitous in Go, it is well advised that you read the official blog post: blog.golang.org/go-slices-usage-and-internals to gain a comprehensive understanding. However, to follow along, it is sufficient to realize* that:

A slice references a fixed-size “underlying” array which actually holds the data.
The fixed-size “underlying” array cannot hold more data than it’s capacity.
If you want to fit more data than the capacity allows, you have to create a brand new “underlying” array with a larger capacity. Then you need to copy the data from the previous array to the new array. The append function does that process behind-the-scenes.
The make(t Type, size int, cap int) Type function allows you to create a slice and preset the array’s capacity in advance.

All slice tricks revolve around managing the capacity or minimizing/avoiding the need to copy data to a new array.

Our Slice

type SeriesInt64 struct {
   values   []int64
}

Copying

func (s *SeriesInt64) copy() *SeriesInt64 {    if len(s.values) == 0 {
        return &SeriesInt64{
            values:[]int64{},
        }
    }    // Copy slice
    x := s.values[0 : len(s.values)]
    newSlice := append(x[:0:0], x...)    return &SeriesInt64{
        values: newSlice,
    }
}

Inserting

func (s *SeriesInt64) insert(row int, val int64) {
    s.values = append(s.values, 0)
    copy(s.values[row+1:], s.values[row:])
    s.values[row] = val
}

In this function, row is the index position you want val to go. If row is between 0 and len-1, then all existing values from row onwards are pushed back by 1.

Appending

func (s *SeriesInt64) append(val int64) {
   row := len(s.values)
   s.insert(row, val)
}

To append a value, use row equal to the length of the slice.

Removing

func (s *SeriesInt64) remove(row int) {
    s.values = append(s.values[:row], s.values[row+1:]...)
}

Prepending

func (s *SeriesInt64) prepend(val int64) {
    if cap(s.values) > len(s.values) {
        s.values = s.values[:len(s.values)+1]
        copy(s.values[1:], s.values)
        s.values[0] = val
        return
    }    // No extra capacity so a new slice needs to be allocated:
    s.insert(0, val)
}

Prepending is potentially more cumbersome. If the underlying array has sufficient capacity to fit the new value, we can copy all existing values by 1 spot, and then “update” the 0ᵗʰ row with val.