使用golang读取yaml配置文件并访问数据

Author

Reads 295

Close-up of hands typing on a laptop with code on screen, perfect for work from home and tech themes.
Credit: pexels.com, Close-up of hands typing on a laptop with code on screen, perfect for work from home and tech themes.

Using Golang to read YAML configuration files is a common task, and we can do it easily with the `viper` package.

The `viper` package allows us to read YAML files using the `ReadConfig` function.

In Golang, we can read a YAML file using the `ReadConfig` function from the `viper` package, as shown in the example code.

This function returns a `Config` object that we can use to access the data in the YAML file.

Syntax

YAML uses indentation and whitespaces to denote the structure of the data.

YAML uses a colon to denote key-value pairs, as seen in the example where "name" and "age" are key-value pairs.

In YAML, lists are denoted with a hyphen, as shown in the example where "hobbies" is a list with three items.

Indentation is crucial in YAML to denote the structure of the data, just like in the example where the "name" and "age" key-value pairs are indented under the top-level key.

Parsing Files

Credit: youtube.com, #31 Golang - Structured Configuration Management with YAML

Parsing files is an essential part of working with YAML in Golang. The yaml.Unmarshal function is used to unmarshal YAML files into structs or maps.

You can use the yaml.Unmarshal function to parse a YAML file into a struct, such as Blog, that matches the structure of the YAML file. The yaml.Unmarshal function takes two arguments: the YAML data and a pointer to the variable that will store the result.

To parse a YAML file into a struct, you need to define a struct that matches the structure of the YAML file. The yaml struct tags are used to indicate the corresponding field names in YAML. Here's an example of how to define a struct Config with fields that match the structure of the YAML data.

Here are some common types of YAML data that can be parsed into maps:

  • map[interface{}]interface{}
  • map[string]interface{}

These types of maps can hold any YAML data, including nested structs and arrays.

Credit: youtube.com, Golang Deep Dive P1 - L53 - Write and Read YAML Format

To parse a YAML file into a map, you can use the yaml.Unmarshal function to unmarshal the YAML data into a map[string]interface{}. This is useful when the structure of the YAML file is not known.

Here's an example of how to access individual nested fields of a YAML file using a map:

  • Use a type assertion to convert the value to the appropriate type.
  • Use the keys of the map to access the nested fields.

For example, to access the "city" field in the nested "address" field, you would use a type assertion to convert the value to a map[string]interface{} and then access the "city" field using the key.

Unmarshaling

Unmarshaling is a crucial step in reading YAML files in Go. It decodes the first document found within a byte slice and assigns decoded values into an out value.

The out parameter must not be nil, and maps and pointers are accepted as out values. If an internal pointer within a struct is not initialized, the yaml package will initialize it if necessary for unmarshalling the provided data.

Credit: youtube.com, Unlocking YAML with Go: How to Unmarshal Structs While Keeping Comments

The type of the decoded values should be compatible with the respective values in out. If one or more values cannot be decoded due to a type mismatch, decoding continues partially until the end of the YAML content, and a *yaml.TypeError is returned with details for all missed values.

Struct fields are only unmarshalled if they are exported (have an upper case first letter), and are unmarshalled using the field name lowercased as the default key. Custom keys may be defined via the "yaml" name in the field tag.

Unmarshaling can also be done using the UnmarshalStrict function, which is like Unmarshal except that any fields that are found in the data that do not have corresponding struct members, or mapping keys that are duplicates, will result in an error.

The Decode function of the *Decoder type also reads the next YAML-encoded value from its input and stores it in the value pointed to by v.

Error Handling

Credit: youtube.com, yaml解析出错,可以给viper提bug了?

Error handling is a crucial aspect of working with YAML in Go. Pretty formatted errors provide extra information on the location of the error from the source YAML document.

This feature makes it easier to find and fix errors. Error messages can also be colorized for better readability.

If you want more control over the output, consider using yaml.FormatError, which accepts two boolean values to turn these features on or off.

Intriguing read: Golang Create Error

Type Error

A TypeError can be a frustrating error to encounter, but understanding its causes can help you troubleshoot and resolve the issue more efficiently.

A TypeError is returned by Unmarshal when one or more fields in the YAML document cannot be properly decoded into the requested types. This can happen when the type of a field in the YAML document doesn't match the type of the variable assigned to it.

The good news is that even if a TypeError occurs, the value is still unmarshaled partially. This means you can still access some of the data, even if not all of it.

Close-up Photo of Document Files
Credit: pexels.com, Close-up Photo of Document Files

The key to resolving a TypeError is to inspect the YAML document and the code that's trying to unmarshal it. By comparing the types of the fields in the YAML document with the types of the variables assigned to them, you can identify the mismatch and make the necessary corrections.

Pretty Formatted Errors

Pretty Formatted Errors can be a game-changer for debugging. Error values produced during parsing contain extra information on the location of the error from the source YAML document, making it easier to find the error location.

This extra information is a default feature that provides valuable context. By default, error values contain this extra information.

You can also control the output by using yaml.FormatError, which accepts two boolean values to control turning these features on or off. This gives you flexibility in customizing the error output.

Go Support

Go has excellent support for reading YAML files.

This support is particularly useful for projects that require data to be stored in YAML format.

You can use Go to generate YAML output, as demonstrated in an example that generates the following output:

A different take: Golang Go

Compatibility

From above crop faceless male developer in black hoodie writing software code on netbook while working in light studio
Credit: pexels.com, From above crop faceless male developer in black hoodie writing software code on netbook while working in light studio

The yaml package in Go supports most of YAML 1.1 and 1.2, including anchors, tags, and map merging.

This is a significant advantage, as it allows you to work with a wide range of YAML files.

Go Struct Tags

Go struct tags are additional metadata information inserted into struct fields, which can be acquired through reflection. They usually provide instructions on how a struct field is encoded to or decoded from a format.

Some popular packages that use struct tags include gopkg.in/yaml.v2, encoding/json, and encoding/xml.

Most of the time, programmers should create application variable names in a more meaningful way, which might differ from the property key. The YAML struct tags play a significant role in filling this gap.

Here are some examples of packages that use struct tags:

  • gopkg.in/yaml.v2
  • encoding/json
  • encoding/xml

Note that the variable names in the struct should start with a capital letter to indicate PUBLIC accessibility, as the YAML parser will use the go-reflect package to parse the tags and map the values.

Implementation

Credit: youtube.com, golang viper tutorial - read config yaml, toml, env

To implement YAML parsing in Go, you'll need to use the YAML parser provided by the Go standard library. Run the command to add the YAML package to your workspace.

The YAML parser relies on the go-reflect package to parse the tags and map the values. Make sure the properties to be parsed have PUBLIC accessibility, with variable names starting with a capital letter.

Use os.ReadFile to read the config file and then parse the content using the YAML API. The yaml.Unmarshal function accepts the YAML content in byte format and the reference of the type for mapping the values.

You might enjoy: Golang vs Go

New Decoder in v2.1.0

The new decoder in v2.1.0 is a game-changer for reading YAML-encoded data.

NewDecoder returns a new decoder that reads from a given reader, r. This decoder introduces its own buffering, allowing it to read data from r beyond the requested YAML values.

The decoder is a key component in the decoding process, and it's essential to understand how it works.

The decoder is used in conjunction with the Decode function, which reads the next YAML-encoded value from its input and stores it in the value pointed to by v. This process is similar to the conversion of YAML into a Go value, as described in the Unmarshal documentation.

Configuration File Includes Complex Map and Slice Types

Woman in focus working on software development remotely on laptop indoors.
Credit: pexels.com, Woman in focus working on software development remotely on laptop indoors.

If you need to parse a YAML file with complex map and slice types, you can use the yaml.Unmarshal function in combination with a map[interface{}]interface{}.

In Go, structs are not the only data structure that can be used to parse YAML files, maps can also be used. This is shown in an example where a YAML file is parsed into a map.

To handle all types of YAML, it's recommended to use a map[interface{}]interface{} instead of a struct. This will be covered in a future example.

Encoding and Decoding

The NewDecoder function is used to create a new decoder that reads from a given reader r.

It introduces its own buffering and may read data from r beyond the YAML values requested.

The Decode function is used to read the next YAML-encoded value from its input and store it in the value pointed to by v.

This function is similar to the Unmarshal function in terms of conversion of YAML into a Go value.

Credit: youtube.com, 【全网最细】一次彻底搞懂YAML分流规则:参数讲解+添加规则+故障排查

To control marshal/unmarshal behavior, you can use the yaml tag.

If both yaml and json tags exist, the yaml tag will take precedence.

Implementing custom marshal/unmarshaling can be done by implementing either the Bytes or Interface variant of marshaler/unmarshaler.

The main difference between these two is that BytesMarshaler/BytesUnmarshaler behaves like encoding/json, while InterfaceMarshaler/InterfaceUnmarshaler behaves like gopkg.in/yaml.v2.

Using the InterfaceMarshaler is better performance-wise when repeatedly marshaling complex objects.

However, if you are providing a choice between a config file format that is read only once, the BytesMarshaler may be easier to code.

Here's an interesting read: Golang Read Json File

Accessing Data

You can access individual fields of a YAML file using structs, which allows you to reference fields by their keys. This is shown in Example 1, where the phone numbers in the YAML file are defined as a slice of PhoneNumber structs.

To access nested fields in a struct, you can use the dot notation, as demonstrated in Example 1. This is useful when dealing with complex data structures.

Expand your knowledge: Golang Array of Structs

Credit: youtube.com, #13 Golang - Creating Custom YAML Serializers with GORM

You can also access individual fields or elements of the data by referencing them by their keys in a map, similar to accessing data using structs. This is shown in Example 2, where the "city" field in the nested "address" field is accessed using a type assertion.

Type assertions are necessary when accessing nested fields in a map, as shown in Example 2. This is because the nested field may be of a different type, and the type assertion helps to determine its type.

To parse a YAML file into a struct, you can use the yaml.Unmarshal function, as shown in Example 3. This requires defining a struct with fields that match the structure of the YAML data.

If the structure of the YAML is not known, it's recommended to use map[interface{}]interface{} instead of a struct, as mentioned in Example 3. This allows for more flexibility when handling different types of YAML data.

You can also parse a YAML file into a map without using a struct, as shown in Example 4. This approach is useful when dealing with complex YAML data.

When parsing YAML data into a map, you can use the yaml.Unmarshal function to store the result in the map, as demonstrated in Example 5. This allows you to access the parsed data using the map's keys.

Parsing YAML data into a map can also be done by reading the YAML file into a variable and then using the yaml.Unmarshal function to parse the data into the map, as shown in Example 6. This approach is useful when dealing with large YAML files.

You might enjoy: Golang Copy Struct

Claire Beier

Senior Writer

Claire Beier is a seasoned writer with a passion for creating informative and engaging content. With a keen eye for detail and a talent for simplifying complex concepts, Claire has established herself as a go-to expert in the field of web development. Her articles on HTML elements have been widely praised for their clarity and accessibility.

Love What You Read? Stay Updated!

Join our community for insights, tips, and more.