sia.hackernoon.com

Optimize String Usage in Go: Efficient Strategies for Combining and Converting with Low Memory

In the world of Go programming, the fmt.Sprintf function is often a go-to because of its easy syntax and flexibility in formatting different data types. But that ease comes at a price – extra CPU overhead and memory allocations that aren’t always ideal, especially when the function gets called repeatedly in loops or in performance-critical parts of your code.

This article talks about why fmt.Sprintf sometimes "breaks your wallet", what alternatives are available (like direct concatenation and strings.Builder), and when those alternatives might be better. Plus, we include some benchmarks to show the performance differences.

Why Does fmt.Sprintf Seem Wasteful?

Even though fmt.Sprintf is easy to use, there are some performance aspects you need to keep in mind:

Parsing Format Overhead: Every call tofmt.Sprintf requires parsing the format string to find placeholders. This process adds extra CPU load.

Type Conversion and Reflection Usage: Since arguments are passed asinterface{}, the function has to do type conversion, and sometimes, use reflection, making it slower than more specific methods.

Memory Allocation: The dynamic formatting process often needs extra memory allocation. When called repeatedly (like in a loop), these small allocations can add up and hurt performance.

Alternative Solutions for Combining Strings

There are several alternatives that can help reduce the overhead of fmt.Sprintf:

1. Direct Concatenation with the `+` Operator

The simplest way to combine strings is to use the + operator. For example:

import "strconv"

value := 123
result := "Value: " + strconv.Itoa(value)

When It’s Better:

For simple operations outside of loops or when the amount of concatenated data isn’t huge.

When code clarity is a priority and performance isn’t critically important.

Advantages:

Clean and easy-to-read syntax.

The Go compiler can optimize simple concatenation.

Disadvantages:

Not efficient in big loops because it creates many new strings and repeated memory allocations.

Example Usage:

func StringConcatenation(a, b string, i int) string {
    return a + b + strconv.Itoa(i)
}

func StringBuilder(a, b string, i int) string {
    var sb strings.Builder
    sb.WriteString(a)
    sb.WriteString(b)
    sb.WriteString(strconv.Itoa(i))
    return sb.String()
}

func fmtSprintf(a, b string, i int) string {
    return fmt.Sprintf("%s%s%d", a, b, i)
}

func StringsJoin(a, b string, i int) string {
    return strings.Join([]string{a, b, strconv.Itoa(i)}, "")
}

Benchmark Results:

BenchmarkStringConcatenation-20   46120149    27.43 ns/op    7 B/op   0 allocs/op
BenchmarkStringBuilder-20         17572586    93.52 ns/op   62 B/op   3 allocs/op
BenchmarkFmtSprintf-20             9388428   128.20 ns/op   63 B/op   4 allocs/op
BenchmarkStringsJoin-20           28760307    70.22 ns/op   31 B/op   1 allocs/op

Direct concatenation with "+" performs best, boasting the fastest execution time (27.43 ns/op) and no extra memory allocations (0 allocs/op, 7 B/op). Conversely, fmt.Sprintf is slowest (128.20 ns/op) with most memory usage (4 allocs/op, 63 B/op). strings.Join is quicker than fmt.Sprintf (70.22 ns/op, 1 allocs/op, 31 B/op), making it a viable option.

2. Using strings.Builder

The strings.Builder package is made to build strings more efficiently by reducing repeated memory allocations.

import (
    "strconv"
    "strings"
)

value := 123
var sb strings.Builder
sb.WriteString("Value: ")
sb.WriteString(strconv.Itoa(value))
result := sb.String()

When It’s Better:

Very suitable for loops or when you need to combine many string pieces.

When you want to lower the number of memory allocations significantly.

Advantages:

Reduces allocation overhead by using a single buffer.

Faster than direct concatenation in repetitive string-building scenarios.

Disadvantages:

A bit more verbose than using the + operator.

Might be overkill for very simple string concatenations.

Example with Words:

var (
    words [][]string = [][]string{
        {"hello", "world", "apple", "canon", "table"},
        {"table", "apple", "world", "hello", "canon"},
        {"canon", "world", "table", "apple", "hello"},
        {"apple", "canon", "hello", "world", "table"},
        {"world", "table", "canon", "hello", "apple"},
        {"hello", "apple", "world", "canon", "table"},
    }
)

func StringConcatenationWithWords(a, b string, i int) string {
    result := a + b + strconv.Itoa(i)
    for _, word := range words[i] {
        result += word
    }
    return result
}

func StringBuilderWithWords(a, b string, i int) string {
    var sb strings.Builder
    sb.WriteString(a)
    sb.WriteString(b)
    sb.WriteString(strconv.Itoa(i))
    for _, word := range words[i] {
        sb.WriteString(word)
    }
    return sb.String()
}

func fmtSprintfWithWords(a, b string, i int) string {
    result := fmt.Sprintf("%s%s%d", a, b, i)
    for _, word := range words[i] {
        result += word
    }
    return result
}

func StringsJoinWithWords(a, b string, i int) string {
    slice := []string{a, b, strconv.Itoa(i)}
    slice = append(slice, words[i]...)
    return strings.Join(slice, "")
}

Benchmark Results:

bashCopyBenchmarkStringConcatenationWithWords-20  3029992   363.5 ns/op   213 B/op   6 allocs/op
BenchmarkStringBuilderWithWords-20        6294296   189.8 ns/op   128 B/op   4 allocs/op
BenchmarkFmtSprintfWithWords-20           2228869   472.1 ns/op   244 B/op   9 allocs/op
BenchmarkStringsJoinWithWords-20          3835489   264.4 ns/op   183 B/op   2 allocs/op

Based on the data, strings.Builder excels in string concatenation, offering the fastest execution time (189.8 ns/op) and minimal memory usage (4 allocs/op, 128 B/op). Direct concatenation is slower (363.5 ns/op, 6 allocs/op, 213 B/op) and less efficient for repeated tasks.

fmt.Sprintf performs worst (472.1 ns/op, 9 allocs/op, 244 B/op), while strings.Join outperforms fmt.Sprintf, yet is still less efficient than strings.Builder.

Alternative Solutions for Converting to String

Besides combining strings, there are also more efficient ways to convert values to strings without using fmt.Sprintf. For simple conversions, the strconv package offers specialized functions that are much faster and use less memory. For instance, to convert an integer to a string, you can use strconv.Itoa:

import "strconv"

func ConvertIntToString(i int) string {
    return strconv.Itoa(i)
}

For other data types, there are similar functions available:

Float: Use strconv.FormatFloat to convert a float to a string. You can adjust the format, precision, and bit size as needed.

goCopyimport "strconv"

func ConvertFloatToString(f float64) string {
    // 'f' is the format, -1 for automatic precision, and 64 for float64
    return strconv.FormatFloat(f, 'f', -1, 64)
}

Bool: Use strconv.FormatBool to convert a boolean to a string.

goCopyimport "strconv"

func ConvertBoolToString(b bool) string {
    return strconv.FormatBool(b)
}

Int64 and Uint64: For larger integer types, use strconv.FormatInt and strconv.FormatUint.

goCopyimport "strconv"

func ConvertInt64ToString(i int64) string {
    return strconv.FormatInt(i, 10) // Base 10 for decimal
}

func ConvertUint64ToString(u uint64) string {
    return strconv.FormatUint(u, 10)
}

Advantages of Using strconv:

Better Performance: The strconv functions are made specifically for type conversion, so they avoid the extra overhead of format parsing found in fmt.Sprintf.

Memory Efficiency: They typically do less memory allocation because they perform a direct conversion without complex formatting.

Clear and Specific Code: Each function serves a specific purpose, making your code easier to read and understand.

For example, here are some simple benchmarks comparing strconv and fmt.Sprintf for various types:

goCopyfunc BenchmarkConvertIntToString(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = strconv.Itoa(12345)
    }
}

func BenchmarkFmtSprintfInt(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = fmt.Sprintf("%d", 12345)
    }
}

func BenchmarkConvertFloatToString(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = strconv.FormatFloat(12345.6789, 'f', 2, 64)
    }
}

func BenchmarkFmtSprintfFloat(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = fmt.Sprintf("%f", 12345.6789)
    }
}

func BenchmarkConvertBoolToString(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = strconv.FormatBool(true)
    }
}

func BenchmarkFmtBoolToString(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = fmt.Sprintf("%t", true)
    }
}

func BenchmarkConvertUintToString(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = strconv.FormatUint(12345, 10)
    }
}

func BenchmarkFmtSprintfUint(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = fmt.Sprintf("%d", 12345)
    }
}

And the results:

BenchmarkConvertIntToString-20          67305488                18.15 ns/op            7 B/op          0 allocs/op
BenchmarkFmtSprintfInt-20               22410037                51.15 ns/op           16 B/op          2 allocs/op
BenchmarkConvertFloatToString-20        16426672                69.97 ns/op           24 B/op          1 allocs/op
BenchmarkFmtSprintfFloat-20             10099478               114.1 ns/op            23 B/op          2 allocs/op
BenchmarkConvertBoolToString-20         1000000000               0.1047 ns/op          0 B/op          0 allocs/op
BenchmarkFmtBoolToString-20             37771470                30.62 ns/op            4 B/op          1 allocs/op
BenchmarkConvertUintToString-20         84657362                18.29 ns/op            7 B/op          0 allocs/op
BenchmarkFmtSprintfUint-20              25607198                49.00 ns/op           16 B/op          2 allocs/op

These benchmarks demonstrate that strconv offers faster execution and uses less memory than fmt.Sprintf for converting values to strings. Thus, for basic conversions (such as int, float, or bool), strconv is an excellent choice when complex formatting isn't required.

Conclusion

In this article, we went through various methods for combining and converting strings in Go—from fmt.Sprintf to direct concatenation with the + operator, strings.Builder, and strings.Join. Benchmarks show that for simple concatenation, the + operator works best, while strings.Builder and strings.Join are optimal for more complex or iterative scenarios. Also, using the strconv package for type conversion (like int, float, bool) is far more efficient than using fmt.Sprintf.

We hope this article gives you a good idea of how to optimize your string handling in Go. Feel free to drop comments or share your experiences. Let’s collaborate and improve our Go code together!

fmt.Sprintf: Looks Simple But Will Burn A Hole in Your Pocket