While developing modern applications, one cannot ignore the fact that the right configuration options make a key difference in how the programs will behave in the long run. This is especially important if you want to fine-grain your application performance or behavior when running in different environments like development, staging or production. What is also useful is understanding the different ways you can pass configuration options to your application, along with the pros and cons of each approach.
In this post, I will explain five different ways to do this, using a base Config object with two properties that I would like to apply:
type Config struct {
ListenAddr string
ListenPort number
}
1. From parameter substitution
This is the case when you have a templated project structure plus some placeholders in the source code for configuration variables that can be substituted on a pre- or post-build phase. For example, tools like Cookiecutter can bake configuration values from a predefined dictionary when they parse code contained within double curly chars {{ }}.
Consider the example below:
func NewConfig() *Config {
return &Config{
ListenAddr: "{{ cookiecutter.listen_addr }}",
ListenPort: "{{ cookiecutter.listen_port }}",
}
}
func main() {
c := NewConfig()
fmt.Println(c)
}
The cookiecutter parameters are substituted before the application is compiled; thus it will be available on startup.
Pros: You can pre-fill some configuration at build time or as part of a CI pipeline without doing extra work.
Cons: Error-prone. It has very specific use cases, and it only works before the code is actually compiled. It is also of limited use at build stage. It’s best practice to apply configuration dynamically from the environment.
2. From the command line
Another popular and more obvious way to pass configuration is through the command line when the application first starts. Most programming languages offer this approach by accepting argument parameters (which is one or more string values passed from the standard input onto the main function, or when the first procedure starts the application). Consider the following example in Go:
func init() {
flag.StringVar(&listenAddr, "listen-addr", "0.0.0.0", "Listen Address")
flag.StringVar(&listenPort, "listen-port", "8080", "Listen Port"
}
func main() {
fmt.Println(listenAddr)
fmt.Println(listenPort)
}
The init function runs before the main, and it reads the parameters, if any, from the command line. If you execute this program without any command line arguments, it will print “0.0.0.0” and “8080” which are the default parameters. In case you specify a parameter like -listen-addr=”127.0.0.1”, it will override the default address and will print “127.0.0.1” and “8080”.
Pros: A reasonable approach. It’s used together with passing configuration params from a file in order to override some specific or some existing config values.
Cons: It is only available during startup of the application. Most of the configuration values need to be known beforehand in order to be applied.
3. From a file
One very common case when setting configuration parameters for your application is to specify a file that will provide a list of config values. Many applications offer this approach, as the file itself can serve as documentation describing in detail each of the parameters’ default functionality.
In addition, you can use different file formats—for example YAML or TOML, which are more readable and human-friendly.
As an example, given a JSON file with the following contents:
{
"listen_addr": "127.0.0.1",
"listen_port": "443",
}
The application can read the values as:
type Config struct {
ListenAddr string `json:"listen_addr"`
ListenPort string `json:"listen_port"`
}
func ConfigFromFile(path string) (Config, error) {
c := Config{}
f, err := os.Open(path)
if os.IsNotExist(err) {
log.Println("config file does not exist")
return c, nil
}
if err != nil {
return c, err
}
if err := json.NewDecoder(f).Decode(&c); err != nil {
return c, err
}
return c, nil
}
Pros: Multiple file formats are available (for example: JSON, XML, YAML). It is useful for providing default values. The file can serve as documentation. Files can be read on demand if needed.
Cons: Config files need to be secured and administered. This exposes the possibility of a file read error, so the application developer should still provide secure default config values.
4. From environment variables
As per the 12-Factor App guidelines, it’s a best practice to store config in the environment, especially for values that vary between deploys. For example, it makes sense to store secrets or sensitive keys in the environment in a way in which they are not easily exposed accidentally in a version control system.
Let’s see an example. With a Bash initialization script:
#/bin/bash program=example export LISTEN_ADDR=127.0.0.1 export LISTEN_PORT=443 exec "$program" "$@"
You can read the environment variables like that:
type Config struct {
ListenAddr string `json:"listen_addr"`
ListenPort string `json:"listen_port"`
}
func ConfigFromFile(path string) (Config, error) {
return Config{
ListenAddr: os.Getenv("LISTEN_ADDR"),
ListenPort: os.Getenv("LISTEN_PORT")
}
}
Pros: A recommended approach. It’s used specifically for storing environment secrets and keys. Suitable also for dynamic deployment options.
Cons: Care has to be taken not to expose environment values in version control. Usually tied to an operating system format.
5. From a key-value store
Last but not least, you can obviously store configuration values in a database or a specialized key-value store. This option is more popular with microservices architecture as it helps maintain the config state between services in a more resilient and scalable way. Of course, this means that extra servers have to be provisioned and maintained in advance.
Let’s see an example using etcd, a famous key-value store used in many open source stacks and in production by many companies. Assuming you have set up a etcd cluster, you can easily set or get config values on demand:
cfg := client.Config{
Endpoints: []string{"http://127.0.0.1:2379"},
Transport: client.DefaultTransport,
HeaderTimeoutPerRequest: time.Second,
}
c, err := client.New(cfg)
if err != nil {
log.Fatal(err)
}
keyApi := client.NewKeysAPI(c)
resp, err := keyApi.Set(context.Background(), "listenAddr", "127.0.0.1", nil)
if err != nil {
log.Fatal(err)
} else {
log.Printf("Set is done. Metadata is %q\n", resp)
}
resp, err = keyApi.Get(context.Background(), "listenAddr", nil)
if err != nil {
log.Fatal(err)
} else {
log.Printf("Get is done. Metadata is %q\n", resp)
// Assign value retrieved here
listenAddr := resp.Node.Value
}
As you can see, there are multiple paths where the assignment and retrieval of the config value can go wrong, and implementors should be aware.
Pros: The most dynamic approach. It works well with storing sensitive config in distributed systems and microservices.
Cons: It’s more difficult to set up and maintain as it requires extra servers and resources. Care has to be taken in order to provide secure defaults in case of a network partition or database failure.
Conclusion
Having different ways to pass configuration options is a good thing, as it gives flexibility and customizability. On the other hand, it’s equally important to use a consistent and managed way of passing those options to prevent configuration mismatch, which can lead to failures or other catastrophic scenarios—meaning that you have to ultimately seal any changes to your production code once deployed, and observe any changes. This will make your infrastructure more stable and immutable, with a consistent behavior across different deployment environments.
Resources
Go Flags
Etcd Documentation
