How do I set custom headers and user agents in Go HTTP requests?

In Go, you can set custom headers and user agents for HTTP requests by modifying the http.Request object before sending the request using the http.Client. The http.Request object has a Header field which is a map of the HTTP header keys to their respective values.

Here's how you can set custom headers and user agents in a Go HTTP request:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    // Create a new HTTP client
    client := &http.Client{}

    // Create a new HTTP request
    req, err := http.NewRequest("GET", "http://example.com", nil)
    if err != nil {
        fmt.Println("Error creating request:", err)
        return
    }

    // Set a custom User-Agent
    req.Header.Set("User-Agent", "MyCustomUserAgent/1.0")

    // Set any other custom headers
    req.Header.Set("X-Custom-Header", "MyValue")

    // Perform the request
    resp, err := client.Do(req)
    if err != nil {
        fmt.Println("Error making request:", err)
        return
    }
    defer resp.Body.Close()

    // Process the response...
}

In the above example:

• We create an http.Client which is responsible for making the request.
• We create a new http.Request using http.NewRequest and specify the method, URL, and the body (if any). In this case, we're making a GET request to http://example.com and the body is nil because GET requests typically don't have a body.
• We then set a custom User-Agent header using req.Header.Set. Replace "MyCustomUserAgent/1.0" with your desired user-agent string.
• We also add a custom header "X-Custom-Header" with the value "MyValue". You can add as many custom headers as you need in a similar way.
• We perform the HTTP request using client.Do(req), which returns an http.Response and an error. If the error is not nil, it indicates a problem with making the request.
• Finally, we defer the closing of the response body to free resources once we're done processing the response.

Remember to replace "http://example.com" with the actual URL you intend to request, and set the appropriate headers and user-agent as per your requirements. Always ensure that you comply with the terms of service and privacy policies of the websites you are scraping or interacting with programmatically.