Golang Plugin Development Essentials

Golang plugin development is a powerful way to extend the functionality of your Go applications.

Plugins can be dynamically loaded and unloaded, allowing you to add or remove features as needed.

To create a plugin in Go, you'll need to define a plugin interface and a plugin binary.

The plugin interface is usually defined using a Go interface, such as `Plugin` in the Go standard library.

A fresh viewpoint: Golang vs Go

Plugins in Go are dynamically linked libraries that can be loaded at runtime, allowing developers to extend the functionality of their programs without recompiling the original code.

A Go plugin is essentially a compiled Go package that can be loaded into a Go program using the plugin API. This API provides a way to discover, load, and use plugins at runtime.

Plugins are stored in a separate file with a .so extension on Unix-like systems or a .dll extension on Windows. This file contains the compiled plugin code and metadata that allows the Go runtime to load and link it correctly.

Discover more: Golang Restful

In the world of Go plugins, the type is a fundamental concept. A plugin is essentially a loaded Go plugin.

Plugins in Go are designed to be loaded and executed at runtime, making them a powerful tool for extending the functionality of your Go programs.

The "type Plugin" declaration is a simple yet crucial part of defining a plugin in Go. A plugin is a loaded Go plugin.

Here's an interesting read: Golang Go

The Lookup function is a powerful tool in plugin development. It searches for a symbol named symName in a plugin p, reporting an error if the symbol is not found.

A symbol is any exported variable or function, which can be accessed after a plugin is loaded. This is especially useful when working with plugins that export multiple functions or variables.

Lookup is safe for concurrent use by multiple goroutines, making it a reliable choice for multi-threaded applications. This means you can use it in your code without worrying about it causing any issues.

The Symbol type is a pointer to a variable or function, allowing you to access exported package symbols after loading a plugin. For example, if a plugin is defined with exported symbols V and F, you can access them using their Symbol types.

Suggestion: Golang Set Env Variable

To set up your environment for developing plugins, you'll need to use Go, specifically the version used in the Tyk Gateway, which you can determine by checking the go.mod file.

Go is the programming language used to develop plugins, and it's essential to use the same version as the Gateway to ensure compatibility.

To simplify this process and minimize the risk of compatibility problems, we recommend using Go workspaces.

A Go workspace will contain the Tyk Gateway source code, your plugins, and some additional files to ensure build compatibility.

Here's a breakdown of what you'll need to set up a Go workspace:

Using a Go workspace ensures that your plugins are built with the same environment and build flags as the Gateway, which is crucial for compatibility.

Shared libraries can be a great way to extend the functionality of your Go application, but they can also be a source of problems. Go is very picky about keeping the main application and the shared libraries it loads compatible.

If this caught your attention, see: Golang Application

One of the main issues with shared libraries in Go is that they can introduce safety issues. If the versions of the packages used by the main application and the shared library don't match exactly, you may get errors or even memory corruption.

As an example, if you modify a package used by both the main application and a shared library, and then rebuild the main application, you may get an error. This is because Go wants all the versions of all packages in the main application and plugins to match exactly.

Developing plugins can be cumbersome due to the need to rebuild all plugins whenever any common packages change. This can be a heavy burden, especially since plugins are often developed separately from the main application.

Here are some common situations where dependencies might cause issues:

To handle these situations, you can update the code to remove the dependency, or use a version of the dependency that includes a go.mod file. Alternatively, you can use a version of the dependency that is promoted to direct in go.mod, allowing both versions to coexist.

Take a look at this: Golang Use Cases

Plugins in Go provide a dynamic loading feature, allowing you to load and unload them at runtime without disrupting the main program.

This flexibility is a key benefit, enabling you to add or remove functionality without affecting the main program. You can even swap or update plugins independently, giving you the freedom to experiment with different implementations.

Plugins promote a modular design pattern by encapsulating specific functionalities in separate components, enhancing code organization, reusability, and maintainability. This is achieved through the plugin feature's ability to run in isolated namespaces, preventing conflicts between different plugins or the main program.

Here are the key plugin features:

Starting from Go 1.8, developers can write plugins and load them dynamically at runtime, adding flexibility to Go programs.

This feature allows for modular and extensible designs, enabling you to extend your application's functionality without recompiling or redeploying the entire codebase.

Plugins can be loaded and unloaded dynamically at runtime, enabling the addition or removal of functionality without disrupting the main program.

For another approach, see: Wordpress Seo without Plugin

This is a key benefit of using plugins, giving you the freedom to iterate and experiment with different implementations.

Plugins offer a more flexible approach than traditional library imports, which are resolved at build time.

They can be swapped or updated independently, allowing for secure and reliable extensability.

The plugin feature promotes a modular design pattern by encapsulating specific functionalities in separate components.

This enhances code organization, reusability, and maintainability.

Plugins run in their own isolated namespaces, preventing conflicts between different plugins or the main program.

Here's a summary of the benefits of plugins:

A virtual endpoint is a powerful feature that allows you to send a response from your Golang plugin without having to make a round trip to the upstream target.

If you send an HTTP response from the Golang plugin custom middleware, the HTTP request processing is stopped and other middleware in the chain won't be used. This means that analytics records will still be created and sent to the analytics processing flow.

Here are the key effects of sending an HTTP response from the Golang plugin custom middleware:

In the example provided, we built a plugin that sends an HTTP 200 response with a JSON payload containing the current time when the request contains the parameter get_time=1 in the query string. This response was served by the Tyk Golang plugin without reaching the upstream target.

Monitoring instrumentation is a powerful feature of custom plugins in Tyk. All custom middleware implemented as Golang plugins support Tyk's current built-in instrumentation.

Custom plugins can emit events with metadata in a specific format, which is "GoPluginMiddleware:" + Path + ":" + SymbolName. For example, if we're working with a custom plugin, the event name might be "GoPluginMiddleware:/path/to/plugin:SymbolName".

The format for a metric with execution time is similar, but with a .exec_time suffix. This allows for detailed performance monitoring of custom plugins.

Most of the arguments related to monitoring instrumentation are applied only to developer flows, which are used for development and testing purposes.

To use the plugin system, you must take a few high-level steps.

Choose the interfaces you want to expose for plugins. This is the starting point for creating a plugin system.

Implement an interface implementation for each interface you want to expose, which communicates over a net/rpc connection or a gRPC connection, or both. This is the most tedious and time-consuming step, but you can see examples in the examples/ directory.

To create a plugin, you'll need to implement a Plugin implementation that knows how to create the RPC client/server for a given plugin type. This involves creating both a client and server implementation.

Here are the basic steps to use a plugin:

To use a plugin system, you must take the following steps. These are high-level steps that must be done. Examples are available in the examples/ directory.

First, choose the interface(s) you want to expose for plugins. This will determine how the plugin will interact with the main application.

For each interface, you'll need to implement an implementation that communicates over a net/rpc connection or over a gRPC connection or both. You'll have to implement both a client and server implementation.

Create a Plugin implementation that knows how to create the RPC client/server for a given plugin type. This is where the plugin system gets really interesting.

Plugin authors call plugin.Serve to serve a plugin from the main function. This is how the plugin becomes part of the main application.

Plugin users use plugin.Client to launch a subprocess and request an interface implementation over RPC. This is how the plugin is used by the main application.

Here are the steps in a concise list:

Before you start using a plugin, you need to configure it properly. This involves setting up the plugin's settings to match your specific needs.

The first step is to identify the plugin's default settings. As mentioned in the "Plugin Installation" section, most plugins come with default settings that can be adjusted later.

For example, the "Social Sharing" plugin comes with default settings for social media platforms. According to the "Plugin Features" section, these settings include Facebook, Twitter, and LinkedIn.

To change these settings, navigate to the plugin's settings page. As explained in the "Plugin Settings" section, this page is usually found in the WordPress dashboard under the "Plugins" tab.

Once you're on the settings page, look for the option to add or remove social media platforms. The "Plugin Configuration" section states that this option is usually found under the "Social Media" tab.

By following these steps, you can configure the plugin to match your specific needs and make it work for you.

You can implement your own authentication method using a Golang plugin and custom "auth_check" middleware.

The package "github.com/TykTechnologies/tyk/ctx" is used to set a session in the request context, which is something "auth_check"-type custom middleware is responsible for.

Our Golang plugin sends a 403 HTTP response if authentication fails.

To build the plugin, run the following command in the folder containing your plugin project.

Our custom middleware just adds a session to the request context and returns if authentication was successful.

Here are the key points about custom authentication:

Logging from a custom plugin is a straightforward process. You can write log entries to Tyk's logging system by importing the package "github.com/TykTechnologies/tyk/log" and using the exported public method Get(). This method returns a logger object that you can use to log messages.

The code snippet below shows how to do this:

```go

var logger = log.Get()

```

To log a message, you can use the logger object's methods, such as Info(), which logs a message at the info level. For example:

```go

logger.Info("Processing HTTP request in Golang plugin!!")

```

This will log a message with the specified text.

Discover more: Golang Message

To develop a Go plugin, you'll need to compile it to native shared object code, which can then be loaded by Tyk Gateway. This requires familiarity with official Go documentation, specifically the plugin package documentation and the tutorial on getting started with multi-module workspaces.

To implement a plugin, identify your plugin package as main and export the functions and variables you want to share with other packages. For example, you can export a Speaker type and a SpeakerName variable, which will become shared library symbols in the compiled shared library.

We recommend using the Tyk Plugin Compiler docker image to build plugins compatible with official Gateway releases. This tool ensures that compatible flags are used when compiling plugins and works around known Go issues.

Golang plugins are a flexible and powerful way to extend the functionality of Tyk by attaching custom logic written in Go to hooks in the Tyk middleware chain.

The chain of middleware is specific to an API and gets created at API load time.

Tyk Gateway performs an API re-load and loads any custom middleware, injecting them into a chain to be called at different stages of the HTTP request life cycle.

For a quick-start guide to working with Go plugins, start here.

The Open Function is a crucial part of plugin development in Go, allowing you to open a Go plugin.

It's safe for concurrent use by multiple goroutines, making it a reliable choice for handling multiple plugin requests at once.

If a path has already been opened, the existing *Plugin is returned, which means you can reuse existing plugins without having to reopen them.

This can be a significant time-saver, especially in large-scale applications where plugin management is crucial.

To develop a Go plugin, you'll need to identify a plugin package as main. This will allow the package to be compiled into a shared library.

In Go, a plugin package must be identified as main to be compiled into a shared library. This is a fundamental concept in plugin development.

To implement a plugin, you'll need to export package functions and variables as symbols. This will make them accessible and visible to other packages and modules that import them.

Here's an interesting read: Golang Test Main

The official Go documentation provides a tutorial on getting started with multi-module workspaces, which is a great resource for plugin development.

A custom Go plugin development flow involves compiling the plugin to native shared object code, which can then be loaded by Tyk Gateway. This requires using the Tyk Plugin Compiler docker image to ensure compatibility with the official Gateway releases.

The Tyk Plugin Compiler docker image provides a cross-compilation toolchain, Go version, and compatible flags to build plugins. It also works around known Go issues, such as plugin already being loaded when a plugin is loaded.

Here's a summary of the development flow:

By following these steps, you'll be able to develop a Go plugin that can be loaded by Tyk Gateway.

A Symbol is a pointer to a variable or function, and in the context of plugins, it's essential to be able to look up the symbols that have been exported.

You can look up the symbol that has been exported in plugin implementation by using the following syntax.

The syntax is straightforward, but understanding what symbols are being exported is crucial. In Example 2, a plugin defined as may be loaded with the Open function and then the exported package symbols V and F can be accessed.

Build Flags Errors can be a real pain to track down, but they're often caused by something simple: missing or mismatched build flags.

Incorrect build flags can cause the Tyk Gateway to report an error when your API attempts to load the plugin. This usually happens when the plugin is built with different flags than the Gateway.

You can check what's different by using the go version -m command for the Gateway and plugin. This will show you the build tokens, which can help you identify the issue.

Here are some common build flag mismatches to watch out for:

These flags need to match between the Gateway and plugin binaries. If they don't, you'll see an error message.

Plugin troubleshooting can be a real challenge, especially when you're working with a Go plugin.

The first step is to check the plugin's documentation, as it's often a good idea to start by reading the plugin's documentation to understand its usage and configuration.

Verify that the plugin is properly installed and configured, as a misconfigured plugin can lead to all sorts of issues.

The plugin's build flags can also play a role in troubleshooting, as the article notes that the build flags can be used to control the plugin's behavior.

A common issue when using Go plugins is the "plugin not found" error, which can be resolved by checking the plugin's path and making sure it's correctly referenced in the code.

Make sure the plugin's dependencies are properly installed and updated, as outdated dependencies can cause problems with the plugin's functionality.

In some cases, the issue may be related to the Go version, so checking the Go version and ensuring it's compatible with the plugin is also a good idea.

If this caught your attention, see: Golang Install Dependencies

We provide the Tyk Plugin Compiler docker image to build plugins compatible with the official Gateway releases.

The Tyk Plugin Compiler toolchain ensures compatible flags are used when compiling plugins, and works around known Go issues such as https://github.com/golang/go/issues/19004 and https://www.reddit.com/r/golang/comments/qxghjv/plugin_already_loaded_when_a_plugin_is_loaded/.

To load a Tyk Golang plugin from a bundle, you'll need to set "enable_bundle_downloader": true in tyk.conf and specify the base URL with the HTTP server where you place your bundles with Golang plugins.

We provide the Tyk Plugin Compiler docker image, which we strongly recommend is used to build plugins compatible with the official Gateway releases.

The Tyk Plugin Compiler provides a cross compilation toolchain, ensuring that plugins are built with compatible flags.

Using this tool helps work around known Go issues, including a problem where plugins can't be loaded due to a bug in the Go compiler.

These issues include a bug where plugins can't be loaded when a plugin is already loaded, as reported on the Go subreddit.

The Tyk Plugin Compiler also ensures that plugins are built with the correct Go version and flags, such as -trimpath, CC, CGO_ENABLED, GOOS, and GOARCH.

Consider reading: T Golang

Tyk offers a dynamic way to load plugins using bundle instrumentation Plugin Bundles.

The bundle command creates an archive with your plugin, which you can deploy to the HTTP server or AWS S3, and then your plugins will be fetched and loaded from that HTTP endpoint.

To enable plugin bundles downloader in tyk.conf, you need to set "enable_bundle_downloader": true and "bundle_base_url": "http://mybundles:8000/abc".

You'll also need to specify the "custom_middleware_bundle" field in your API spec with the filename of the bundle (.zip archive) to be fetched from the HTTP endpoint.

A ZIP archive with a bundle typically contains two files: AddFooBarHeader.so, the Golang plugin, and manifest.json, a special file with meta information used by Tyk's bundle loader.

Here's an example of what the contents of manifest.json might look like:

Plugins can be a double-edged sword when it comes to security. They may enable high-performance integration of separate parts, but they also come with significant drawbacks.

The ability to dynamically load parts of an application during execution can be a recipe for disaster if not properly secured. Plugins can share data structures directly with applications, creating a potential vulnerability.

Many users decide that traditional interprocess communication (IPC) mechanisms are more suitable despite the performance overheads. This is because plugins can be a security risk if not handled carefully.

Plugins may be a useful building block in some designs, but they require careful consideration during the design phase.

Upgrading your plugins can be a real challenge, especially if you're not familiar with the process. You need to re-compile your plugin with the new version.

Tyk Gateway will try to find a plugin with the name provided in the API definition, and if none is found, it will fall back to searching for a plugin file with a specific naming convention.

To avoid any issues, you should have your plugins compiled for the new version before performing the upgrade. This way, you can ensure a smooth transition.

Additional reading: Golang Version Manager

Tyk v4.1.0 introduced a change that makes this process easier. The compiler now automatically creates plugin files following a certain convention.

This means that when you upgrade, say from Tyk v5.2.5 to v5.3.0, you only need to have the plugins compiled for v5.3.0. This can save you a lot of time and effort in the long run.