In Go, you can use the built-in gzip library together with bytes.Buffer to compress a []byte to obtain a []byte containing gzip-compressed data:

content := "Hello World!"
buf := &bytes.Buffer{}
gzWriter := gzip.NewWriter(buf)
gzWriter.Write([]byte(content))
gzWriter.Close()

This full example compresses the data in-memory and writes it to a file called text.txt.gz afterwards. Note that you can also gzip directly into a file and if you don’t need to do anything else with the compressed data, you might as well have a look at our previous post How to write to gzipped file in Go.

package main

import (
    "compress/gzip"
    "os"
)

func main() {
    content := "Hello World!"
    // Initialize gzip
    buf := &bytes.Buffer{}
    gzWriter := gzip.NewWriter(buf)
    gzWriter.Write([]byte(content))
    gzWriter.Close()
    // Convert buffer to
    ioutil.WriteFile("test.txt.gz", buf.Bytes(), 0644)
}

You can use zcat test.txt.gz to see that the content has been written to the file correctly.

This example shows you how to directly write gzip-compressed data using Go’s gzip library.

First, open the file and use gzip.NewWriter() to create a new io.Writer on it:

// Open file
f, _ := os.Create("test.txt.gz")
defer f.Close()
// Create gzip writer
gzWriter := gzip.NewWriter(f)

Now you can gzWriter.Write(). Don’t forget to gzWriter.Close()

content := "Hello World!"
gzWriter.Write([]byte(content))
gzWriter.Close()

Full example:

package main

import (
    "compress/gzip"
    "os"
)

func main() {
    // Open file
    f, _ := os.Create("test.txt.gz")
    defer f.Close()
    // Create gzip writer
    gzWriter := gzip.NewWriter(f)
    // Write content and close
    content := "Hello World!"
    gzWriter.Write([]byte(content))
    gzWriter.Close()
}

You can use zcat test.txt.gz to see that the content has been written to the file correctly.

This is the minimal example that writes a string (Hello World! in this example) to a file (test.txt in this example) using Go.

package main

import "io/ioutil"

func main() {
    content := "Hello World!"
    ioutil.WriteFile("test.txt", []byte(content), 0644)
}

Note that this example:

• Creates the file with permission mode 0644, i.e. only the owner can write but others can read.
• Encodes the text as UTF-8

io.BytesIO in Python 3.x provides a convenient way of having a file-like object that actually streams to/from memory.

In Go, the equivalent problem is to have a io.Reader and/or and io.Writer (i.e. the equivalent of Python’s file-like object) that is backed by a []byte or a string.

Solution: Use bytes.Buffer !

// Initialize an empty buffer (e.g. for writing)
buf := &bytes.Buffer{}

// Initialize a buffer with a []byte content (e.g. for reading)
myBytes := ...
buf := bytes.NewBuffer(myBytes)

// Initialize a buffer with string
myStr := "test 123"
buf := bytes.NewBufferString(myStr)

This example uses goquery to request a HTML page (https://techoverflow.net) via the Go net/http client and then uses goquery and a simple CSS-style query to select the <title>...</title> HTML tag and print it’s content.

package main

import (
    "fmt"
    "log"
    "net/http"

    "github.com/PuerkitoBio/goquery"
)

func main() {
    // Perform request
    resp, err := http.Get("https://techoverflow.net")
    if err != nil {
        print(err)
        return
    }
    // Cleanup when this function ends
    defer resp.Body.Close()
    // Read & parse response data
    doc, err := goquery.NewDocumentFromReader(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    // Print content of <title></title>
    doc.Find("title").Each(func(i int, s *goquery.Selection) {
        fmt.Printf("Title of the page: %s\n", s.Text())
    })
}

Example output:

Title of the page: TechOverflow

This example shows you a Go HTTP Server that forwards requests to an upstream Server using a HTTP client. Note that this example does not stream the upstream server’s responses but downloads them to RAM and then serves them to the client (hence you might need to improve upon this example if you intend to serve large files).

Also, note that this example does not implement proper error handling for the upstream request since it’s only intended to serve as a basic starting point for you to develop your own reverse proxy implementation.

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

func main() {
    client := &http.Client{}
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        // Perform request
        url := "https://techoverflow.net" + r.URL.Path
        req, _ := http.NewRequest(r.Method, url, r.Body)
        req.Header = r.Header
        resp, _ := client.Do(req)
        // Cleanup when this function ends
        defer resp.Body.Close()
        // Copy headers from request to response
        header := w.Header()
        for name, headers := range resp.Header {
            // Iterate all headers with one name (e.g. Content-Type)
            for _, hdr := range headers {
                header.Add(name, hdr)
            }
        }
        // Read all the response data into a []byte
        body, _ := ioutil.ReadAll(resp.Body)
        // Write header & body
        w.WriteHeader(resp.StatusCode)
        w.Write(body)
    })
    // log.Fatal shows you if there is an error like the
    //  port already being used
    fmt.Println("Listening on port 8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Further reading:

• Go net/http package
• Iterating HTTP client response headers in Go (which we use as a basis to implement our header copying mechanism)

In this example we iterate all the HTTP headers the server sends back after a HTTP request made with the net/http Go library.

for name, headers := range resp.Header {
    // Iterate all headers with one name (e.g. Content-Type)
    for _, hdr := range headers {
        println(name + ": " + hdr)
    }
}

Full example:

package main

import (
    "net/http"
)

func main() {
    // Perform request
    resp, err := http.Get("https://ipv4.techoverflow.net/api/get-my-ip")
    if err != nil {
        print(err)
        return
    }
    // Cleanup when this function ends
    defer resp.Body.Close()
    // Read all the headers
    for name, headers := range resp.Header {
        // Iterate all headers with one name (e.g. Content-Type)
        for _, hdr := range headers {
            println(name + ": " + hdr)
        }
    }
}

This example will print, for example:

Content-Type: application/octet-stream
Content-Type: text/plain charset=UTF-8
Content-Length: 11
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, OPTIONS
Server: nginx/1.14.0 (Ubuntu)
Date: Sat, 16 Nov 2019 00:27:05 GMT

You can use TechOverflow’s IPv4 HTTP API & IPv6 HTTP API within Go:

import (
    "io/ioutil"
    "net/http"
)

func GetExternalIPv4Address() (string, error) {
    // Perform request
    resp, err := http.Get("https://ipv4.techoverflow.net/api/get-my-ip")
    if err != nil {
        return "", err
    }
    // Cleanup when this function ends
    defer resp.Body.Close()
    // Read all the response data into a []byte
    body, err := ioutil.ReadAll(resp.Body)
    // Decode & print
    return string(body), nil
}

func GetExternalIPv6Address() (string, error) {
    // Perform request
    resp, err := http.Get("https://ipv6.techoverflow.net/api/get-my-ip")
    if err != nil {
        return "", err
    }
    // Cleanup when this function ends
    defer resp.Body.Close()
    // Read all the response data into a []byte
    body, err := ioutil.ReadAll(resp.Body)
    // Decode & print
    return string(body), nil
}

Usage example:

package main

func main() {
    v4, _ := GetExternalIPv4Address()
    v6, _ := GetExternalIPv6Address()
    println(v4)
    println(v6)
}

If you save both files in a directory named GoIPAddress, you can run

$ go build
$ ./GOIPAddress
31.190.168.110
2a03:4012:2:1022::1