I recently switched to working with Go as one of the primary languages that I work with on a day to day basis. I’m still getting the hang of it, but so far I’m enjoying it. I love the set of standard tooling that comes with the language. One of the tools, which is usually not included with a language’s standard toolkit is a performance benchmarking tool. Go includes this in it’s go test tool and can be used to go basic performance testing.
I started out with a mini-project to test this out. You can find a basic set of instructions on how to start off with here. However, there is nothing like a real mini-project to really get the hang of it and appreciate it. Hence, I am going to use the benchmarking tools to test the efficiency of an idea I had to speed up a very trivial thing that us programmers do on a daily basis.
The Project
Problem: What’s the most efficient way to get the first element from a list, which could be empty
Seems pretty simple. There are a couple of ways to do this. I can image many people just thinking that there is only 1, which is the 1st one.
- Check of the array is empty or null, if not, then simply return the 0 indexed value.
- Iterate over the array, return the value. The loop will only run once, returning the first element.
I liked 2 for it’s simplicity. It also looked like there were less operations going on since we’re skipping the check on the length or the fact that it’s null. However, list comprehension in modern languages also make list traversal very simple, but just cause they look simple, doesn’t mean they’re efficient! Hence, what better thing to test than this!
Setting Up
I am assuming you already have you Go development environment setup, with a workspace which the GOPATH environment variable points to. For details on how to do so, see here.
Let’s first start by creating a package that will contain our code that we will be testing and benchmarking. To mimic what I would be doing in reality, I created an arbitrary struct FullName. Hence, we’ll be working with a list of structs rather than primitive kinds. I’ve also implemented two funcitons on FullName to pull out the first element in the two different ways that we discussed before.
File: $GOPATH/src/github.com/danishm/firstelement/firstelement.go
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
package firstelement import ( "errors" ) // FullName A structure to store a full name type FullName struct { FirstName string LastName string } // GetFirstTraditional returns the first element by doing // a length check and using the 0 index of an array func GetFirstTraditional(data []FullName) (FullName, error) { if len(data) > 0 { return data[0], nil } return FullName{}, errors.New("Empty list") } // GetFirstRangeLoop returns the first element by using // a range based loop and returning element in 1st loop func GetFirstRangeLoop(data []FullName) (FullName, error) { for _, fn := range data { return fn, nil } return FullName{}, errors.New("Empty list") } |
To test and benchmark this code, we will create another file with sample data, test functions and benchmarking functions. This filename must have _test in it. In this way Go uses convention over config to make things easy and get you up and going quicker.
File: $GOPATH/src/github.com/danishm/firstelement/firstelement_test.go
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
package firstelement import ( "testing" ) // GetSampleData Returns a sample list of FullNames func GetSampleData() []FullName { samples := []FullName{ FullName{"Danish", "Mujeeb"}, FullName{"Mikael", "Danish"}, FullName{"Hello", "World"}, } return samples } // Testing functions func TestGetFirstTraditional(t *testing.T) { samples := GetSampleData() fn, err := GetFirstTraditional(samples) if err != nil { t.Error("Error retrieving first element") } if fn.FirstName != "Danish" { t.Error("Expected first name to be 'Danish'", fn.FirstName) } } func TestGetFirstRangeLoop(t *testing.T) { samples := GetSampleData() fn, err := GetFirstRangeLoop(samples) if err != nil { t.Error("Error retrieving first element") } if fn.FirstName != "Danish" { t.Error("Expected first name to be 'Danish'", fn.FirstName) } } // Benchmarking functions func BenchmarkGetFirstTraditional(b *testing.B) { samples := GetSampleData() for i := 0; i < b.N; i++ { GetFirstTraditional(samples) } } func BenchmarkGetFirstRangeLoop(b *testing.B) { samples := GetSampleData() for i := 0; i < b.N; i++ { GetFirstRangeLoop(samples) } } |
Testing and Benchmarking
Once we have these two files in place, we can use the go test -bench . -v command to test and benchmark. The command will run the unit tests first to help us confirm our implementation and then the benchmarks. Below is the output of the command
|
1 2 3 4 5 6 7 8 9 |
$ go test -bench . -v === RUN TestGetFirstTraditional --- PASS: TestGetFirstTraditional (0.00s) === RUN TestGetFirstRangeLoop --- PASS: TestGetFirstRangeLoop (0.00s) BenchmarkGetFirstTraditional-8 2000000000 0.32 ns/op BenchmarkGetFirstRangeLoop-8 500000000 3.85 ns/op PASS ok github.com/danishm/firstelement 4.294s |
This speaks pretty loudly, the range loop based approach is roughly 12x slower! I was suspecting it be slow, but this was significant. The loop may look classy, but it’s slow. On top of that, it is also known to look weird to a lot of developers.
However, I want to keep on digging in. What if we have a traditional for loop?
Re-Factoring The Loop
Although the range based loop looks nice and clean, I wanted to benchmark using a traditional C/C++ style loop with no iterators. Hopefully, this will perform better. I added the following functions to each of our files.
Add to File: $GOPATH/src/github.com/danishm/firstelement/firstelement.go
|
1 2 3 4 5 6 7 8 |
// GetFirstTraditionalLoop returns the first element by using // a traditional loop and returning element in 1st loop func GetFirstTraditionalLoop(data []FullName) (FullName, error) { for i := 0; i < len(data); i++ { return data[i], nil } return FullName{}, errors.New("Empty list") } |
Add to File: $GOPATH/src/github.com/danishm/firstelement/firstelement_test.go
|
1 2 3 4 5 6 |
func BenchmarkGetFirstTraditionalLoop(b *testing.B) { samples := GetSampleData() for i := 0; i < b.N; i++ { GetFirstTraditional(samples) } } |
Running the benchmark using these new functions yeilded some interesting results
|
1 2 3 4 5 6 |
$ go test -bench . BenchmarkGetFirstTraditional-8 2000000000 0.32 ns/op BenchmarkGetFirstRangeLoop-8 500000000 3.81 ns/op BenchmarkGetFirstTraditionalLoop-8 2000000000 0.32 ns/op PASS ok github.com/danishm/firstelement 5.122s |
The traditional for loop approach is just as fast as the length check and 0-index approach. This just show how much baggage the range keyword comes with. As you can see here, it provides a consistent way to iterate over an array, slice, string, map, or channel. This makes for readable and maintainable code, but it comes at a cost.
I decided to go deeper!
What If We Used Pointers
I created three new versions of the implementation and test function appended with P, which were the pointer based versions of the original functions. You can see the complete code at this github repository. Here are the results of the run
|
1 2 3 4 5 6 7 8 9 |
$ go test -bench . BenchmarkGetFirstTraditional-8 2000000000 0.32 ns/op BenchmarkGetFirstRangeLoop-8 500000000 3.79 ns/op BenchmarkGetFirstTraditionalLoop-8 2000000000 0.31 ns/op BenchmarkGetFirstTraditionalP-8 2000000000 0.32 ns/op BenchmarkGetFirstRangeLoopP-8 1000000000 2.85 ns/op BenchmarkGetFirstTraditionalLoopP-8 2000000000 0.32 ns/op PASS ok github.com/danishm/firstelement 8.092s |
This yields something interesting as well. The pointer version of BenchmarkGetFirstRangeLoop runs around 1.3x faster than the non-pointer version! I believe this is due to the fact that with pointers, you don’t have to copy the whole data structure when you return it. You’re simply passing back the pointer to it, which probably saved some time.
I hope you enjoyed reading this and are encouraged to benchmark and test your code more diligently.