io package in golang
The io package utilizes the Reader and Writer types in the core language of Go.
type Reader interface {
Read(p []byte) (n int, err error)
}type Writer interface {
Write(p []byte) (n int, err error)
}
As you can tell there are a lot of methods having to do with Reading and Writing. This is an easier way of looking at packages. It lets you break down how methods are used on the io package.
We will look at the first method to break down how to utilize the package and use the Reader/Writer types.
The following is an example of the io.copy()
Copy( takes in a Writer and Reader type) returns an int64 and error
func Copy(dst Writer, src Reader) (written int64, err error)
So the Reader type is taken in from the strings.NewReader as a string and you then get the io package methods like io.Copy()
The io.Copy needs a Reader which is “r” and a Writer which is os.Stdout.
The b, err := is what it is going to be saved in memory. So the data of b is saved as the int64 from the return of func Copy. It is important to understand what is returned so that you know what is able to be saved in memory. As you can tell the types are important in determining a lot of things. The more you understand them the more you can utilize the package and utilize them from the ground up. Remember though you are only getting an int64 as a return.
We can take a look now at some of the other methods io has to offer and how they run. The next one will be CopyBuffer. There are a lot of mentions of a Buffer in the go package.
A buffer is a way to store data from a Reader or Writer so that the os does not need to make a bunch of separate operations and it can be utilized to control data more efficiently. You can think of it as when a video buffers but on a more operational level. You can even edit the size of the buffer for moving data. This is editable inside the bufio package.
bufio.NewWriterSize(w, 2)
Even advocates say to utilize this package but understanding the basics of utilizing any package is important.
I believe the first time I had seen this package and learned about bufio I was more worried about hacking these methods together to read data from one file and storing it in a database. I sorta thought of it as a sudo os operations but the more you begin to break apart these types you see why packages came to be.
https://gobyexample.com/reading-files
f, err := os.Open("/tmp/dat")
r4 := bufio.NewReader(f)As you can tell in the example above you are utilizing the bufio package and using the NewReader. So you know you are attempting to save data in memory by reading it. You know by now that you are getting a Reader type and some of its io methods. That r4 is where this memory is saved and can be utilized. One thing you do see after that line in is peek()
b4, err := r4.Peek(5)When you see something you do not know then just look it up in the standard docs.
- It gets satisfied by a Reader type
- It takes in an int
- return a slice of bytes[]
The Peek() method allows you to feel around for lines of data that you are trying to read. It might seem trivial but having a deeper understanding is important. When you read data from a file, the bufio package utilizes bytes and you have to think of it in terms of streams.
So there is a lot going on here that most people pass up. They think well I just got data from a file…
The Open method on the os package needs a string which is the file directory for discovery and the filename. This data is then being read through Read() method that reads the data by Bytes and turns them into slice of byte[]. This happens in streams. You also have to keep in mind the end of the line and the file. This returns an error when done. This is because there might be multiple lines and files. You can wield this by using the bufio methods.
You create a map so that not only is the data parsed in multiple collection style memory but so that you have control of it. If you need to parse this data to another file in a certain order or calculate data this will be useful.
The point of reading into things this deep is so that you can come up with solutions from the ground up and maybe come up with your own package or methods. This also allows for a mental model of your data so that it is more easily understood and used.
So far I have described the following in general use.
Overview
- io package https://golang.org/pkg/io/
- bufio package https://golang.org/pkg/bufio/
- os package https://golang.org/pkg/os/
- An example of generally how these could be used in action. https://gobyexample.com/reading-files
Next is brief look at the http package and a simple server.
