Golang Scheduling in Depth: Understanding the Scheduler and Goroutines

Golang scheduling is a complex topic, but understanding the scheduler and goroutines is key to writing efficient and scalable code.

The Go scheduler is designed to be lightweight and efficient, with a focus on concurrency rather than parallelism. This means that multiple goroutines can run concurrently, but not necessarily at the same time.

Goroutines are lightweight threads that can be created and managed with ease. They're the building blocks of concurrent programming in Go, and are essential for writing scalable and high-performance code.

A goroutine is essentially a function that runs concurrently with other functions, allowing for efficient use of system resources. This is in contrast to traditional threads, which require more overhead and resources to manage.

Additional reading: Golang Source Code

Goroutines are similar to lightweight threads, running concurrently and capable of running in separate threads. They're incredibly lightweight, occupying only a few kilobytes each.

In Go, these coroutines are called Goroutines, and they're derived from the concept of coroutines, allowing a set of reusable functions to run on a set of threads. This approach simplifies programming and offers easier concurrency.

Goroutines are not just separate threads running in parallel on the CPU being managed by the OS, but rather a way to run multiple tasks at the same time. They're dynamically scalable, with the runtime automatically allocating more memory when needed.

A Goroutine's stack may only consume a few kilobytes, but it's dynamically scalable, making it a great way to handle concurrency in Go. This small memory footprint allows Go to support a large number of Goroutines within limited memory space.

There can exist more Goroutines than kernel threads, but there can never be more running threads than cores. This means that Goroutines need to be mapped to kernel threads in order to run on the cores.

In Go, every goroutine starts its execution in the global run queue but gets assigned a local run queue at some point.

Each thread mostly interacts with its own local run queue, picking up a goroutine and executing it, which offloads most of the work to the local run queue.

Broaden your view: Run Golang File

Since the local run queue is only ever accessed by a single thread, it doesn't even need a mutex, making the code simpler.

The global run queue still exists and has a mutex, but a single thread is mostly picking up goroutines from its own local run queue, massively minimizing calls to the global run queue.

Occasionally, a thread might fetch from the global run queue if its own local run queue is empty, but this is a rare case.

The Golang scheduler is a complex beast, but understanding its scheduling scenarios can help you write more efficient code. The GPM model is the core of the scheduler, and it's responsible for scheduling Goroutines.

In the GPM model, a Goroutine is created and added to the local queue of a Processor (P). If the local queue is full, the Goroutine is added to the global queue.

When a Processor is idle, it will check the local queue of another Processor to steal work. This is called the work-stealing mechanism. If the global queue is empty and the local queue is also empty, the Processor will randomly steal work from another Processor.

Each Processor has its own local run queue, which is used to store Goroutines that are ready to be executed. The local run queue has a dedicated length of 256 Goroutines, and if it's full, the Goroutines are added to the global queue.

The Golang scheduler also uses a concept called per-thread state, which is stored in the Processor. This state is used to manage the interaction between the local run queues, the global run queue, and the kernel threads.

In the M:P:N threading model, the M is the number of kernel threads, the P is the number of Processors, and the N is the number of Goroutines. This model is used to manage the interaction between the local run queues, the global run queue, and the kernel threads.

If a Goroutine is waiting for a system call to complete, the Processor will release the associated Processor and transfer it to another available thread for execution. This is called the handoff mechanism.

The Golang scheduler also uses a concept called the network poller, which is responsible for handling asynchronous system calls such as network I/O. If a Goroutine is waiting for a network request to complete, it will be added to the network poller.

Here's an interesting read: Golang State Machine

In the GPM model, a Goroutine is created and added to the local queue of a Processor. If the local queue is full, the Goroutine is added to the global queue. The Processor will then check the local queue of another Processor to steal work.

The Golang scheduler uses a concept called the processor, which is a layer of abstraction that contains variables to execute Go code. Each thread acquires a single processor, and when a thread gets blocked on a system call, it will release the processor and a different thread will acquire it and continue where it left off.

The GPM model also uses a concept called the locality property, which states that a newly created Goroutine is given top priority and is placed into the local queue first. The Goroutines in the local queue will then be scheduled in a sequential manner.

The Golang scheduler uses a concept called the idle P pool, which is used to store Processors that are not currently being used. When a Processor is idle, it will return its Processor to the idle P pool.

In the GPM model, a Goroutine can only run within a Processor, and each Processor must hold a Processor. This is known as the 1:1 relationship between Processors and Processors.

You might like: Azure Scheduler

Context Switching is a critical aspect of the Go scheduler. It allows the scheduler to make scheduling decisions based on well-defined user-space events that occur at safe points in the code.

These events and safe points manifest themselves within function calls. Function calls are critical to the health of the Go scheduler. If you run any tight loops that are not making function calls, you will cause latencies within the scheduler and garbage collection.

There are four classes of events that occur in your Go programs that allow the scheduler to make scheduling decisions. This doesn't mean it will always happen on one of these events. It means the scheduler gets the opportunity.

The use of the keyword go is how you create Goroutines. Once a new Goroutine is created, it gives the scheduler an opportunity to make a scheduling decision.

Garbage collection causes the GC to create a lot of scheduling chaos, but the scheduler is very smart about what a Goroutine is doing and it will leverage that intelligence to make smart decisions.

System calls can also trigger context-switching, and the scheduler can sometimes context-switch a Goroutine off the M and context-switch a new Goroutine onto that same M.

Synchronization and Orchestration operations can also cause the scheduler to context-switch a new Goroutine to run.

Related reading: Golang Switch Fallthrough

The concept of "The" is a fundamental building block of scheduling.

It's used to specify a particular time or event.

In scheduling, "The" is often used to denote a specific point in time, such as "The meeting starts at 2 PM" or "The deadline is Friday at 5 PM".

You can use "The" to refer to a specific event or appointment, making it easier to communicate and avoid confusion.

For example, if you have a meeting with John at 10 AM, you can say "The meeting with John is at 10 AM" to make it clear what you're referring to.

Recommended read: Time.parse Golang

M0 is the main thread that starts after a Go program launches. It's located in the global command runtime.m0 and doesn't require any heap allocation.

M0 is responsible for executing initialization operations and launching the first G. This is a crucial step in the program's lifecycle.

Here are some key facts about M0:

After launching the first G, M0 functions just like any other M, executing Gs and handling scheduling information.

In Go, a Goroutine can occupy the CPU for a maximum of 10ms, preventing other Goroutines from being starved of resources.

This is a distinguishing feature of Goroutines compared to regular Coroutines. It allows for a more efficient use of resources and prevents any one Goroutine from dominating the CPU for too long.

The Go scheduler uses a strategy called Preemption to manage Goroutines. In this strategy, a Goroutine must voluntarily yield the CPU before the next Goroutine can execute.

Readers also liked: Golang Go

The global G queue still exists in the new scheduler, but with weakened functionality. It allows an M to retrieve Goroutines from the global G queue when it cannot find any in the queues of other P's.

Work Stealing is another interesting feature that can be added to the system. Whenever a thread's local run queue is empty, it can try to steal work from other local run queues, helping to balance the load of Goroutines.

This approach ensures that even if a thread is executing a Goroutine for a very long time, other Goroutines in its local run queue would not get starved. Eventually, some other thread would "steal" those Goroutines and execute them.

System calls can take a long time to return, and if a goroutine is performing a syscall, it may block the M, making it unavailable to execute other goroutines. This can lead to a situation where all threads are waiting for system calls to finish.

System calls can be handled asynchronously, allowing the network poller to process them efficiently. This is accomplished by using kqueue (MacOS), epoll (Linux), or iocp (Windows) within their respective OS's.

To prevent goroutines from blocking the M, the Go scheduler can detach the M from the P with the blocking goroutine and bring in a new M to service the P. This allows other goroutines to be executed without creating new Ms.

The network poller is responsible for handling asynchronous system calls such as network I/O, and if a goroutine is waiting for a network request to complete, it will be added to the network poller to avoid blocking the kernel thread.

System calls can be a challenge to handle, especially when they take a long time to return. This can cause threads to sleep and become unproductive, leading to a situation where all threads are waiting for system calls to finish.

Creating a new thread before a system call can be a solution to this problem. This way, the current thread can sleep while the system call is being processed, allowing other threads to continue working.

However, creating too many threads can lead to memory allocation issues and problems with work-stealing. This is because each thread requires its own local run queue, which can become a bottleneck.

A simpler solution is to occasionally check the global run queue instead of the local run queue. This can help to distribute work more efficiently and reduce the likelihood of memory allocation issues.

In some cases, system calls can be made asynchronously using a network poller. This can help to prevent threads from blocking and allow other threads to continue working.

However, not all system calls can be made asynchronously. In these cases, the thread will block and become unproductive until the system call is complete.

The Go scheduler has a mechanism to handle this situation by detaching the blocked thread from its processor and assigning it to a new processor. This allows other threads to continue working while the blocked thread is waiting for the system call to complete.

On a similar theme: Golang Use Cases

In some scenarios, a thread may be able to reacquire its original processor after the system call is complete. However, if this is not possible, the thread will be added to the global run queue and may be stolen by another processor.

Overall, handling system calls requires a careful balance between efficiency and memory allocation. By understanding the different scenarios and mechanisms involved, developers can write more efficient and effective code.

The Go runtime provides a few APIs that can be used to control the scheduler, but I'd recommend using them only when necessary.

One of these APIs is runtime.NumGoroutine(), which returns the number of goroutines that are currently running.

If you change the GOMAXPROCS value while the program is running, it will cause a stop-the-world operation, resizing the array that holds the processors before resuming the program.

The runtime.GOMAXPROCS() API sets the maximum number of processors that can be used by the Go runtime.

You can also use runtime.Gosched() to cause the current goroutine to yield to the processor, adding it to the global run queue.

Another API, runtime.Goexit(), causes the current goroutine to exit without affecting other goroutines.

The runtime.LockOSThread() and runtime.UnlockOSThread() APIs are used to wire a goroutine to the underlying kernel thread, primarily when the goroutine changes the thread state.

Here are some of the runtime APIs and their uses:

Fairness and Preemption are crucial components of the Go runtime's scheduler. The scheduler ensures that no single goroutine hogs a thread for a very long time by pre-emptively pausing it if it takes longer than 10ms.

To achieve fairness, the scheduler gives each goroutine a fair chance to be executed, without giving more time to goroutines that are more active than others. This is because it will lead to starvation of less active goroutines.

The Go runtime uses a hierarchy of fairness to ensure that goroutines are not starved. The hierarchy includes:

Preemption is the ability of the scheduler to preempt (stop) a goroutine that is currently running and execute another goroutine. This is done to make sure that goroutines that are time-blocking don't consume a lot of time and give a chance to other goroutines to be executed. The scheduler uses a 10ms time slice to determine when to preempt a goroutine.

A preempted goroutine will be added to the end of the global run queue, which is a FIFO queue. This means that the goroutine will be executed after all the other goroutines in the global run queue.

For more insights, see: Golang Time Unix

Goroutines can be in one of three states: Runnable, Running, or Blocked. A goroutine is Runnable when it's ready to be executed and waiting for the kernel thread to execute it.

The scheduler keeps track of the goroutines in a run queue, which follows the FIFO principle. This means that goroutines are executed in the order they were added to the queue.

A mutex is used to synchronize access to the run queue, ensuring that only one kernel thread can access it at a time. This is crucial for preventing data corruption and ensuring the stability of the system.

Here are the different states a goroutine can be in:

Goroutines are incredibly lightweight, occupying only a few kilobytes each, which allows Go to support a large number of them within limited memory space.

The scheduler keeps track of goroutines in a run queue, which is a queue of goroutines that are ready to be executed. It follows the FIFO principle, meaning the first goroutine added to the queue is the first one to be executed.

The run queue is a shared resource, so the scheduler uses a mutex to synchronize access to it. A mutex is a lock that ensures only one kernel thread can access the run queue at a time.

The scheduler is responsible for adding goroutines to the run queue and removing them from it. This ensures that goroutines are executed in the correct order.

By using a mutex, the scheduler prevents multiple kernel threads from accessing the run queue simultaneously, which would lead to data corruption and errors.

Every non-dead G has a user stack associated with it, which is what user Go code executes on. These stacks start small, typically around 2K, and grow or shrink dynamically as needed.

User stacks are used for executing user Go code, and their size can vary, but they're generally small to prevent excessive memory usage.

Most functions in Go start with a prologue that inspects the stack pointer and the current G's stack bound, which is done to prevent stack overflows.

The garbage collector does not scan system stacks, which means that memory allocated on these stacks is not subject to garbage collection.

System and signal stacks are fixed in size and cannot grow, but they're large enough to execute runtime and cgo code, typically around 8K in a pure Go binary.

Debugging and analysis are crucial steps in understanding and optimizing Go's scheduling behavior. Execution traces contain a wealth of information about the runtime, including goroutine scheduling actions, time spent in the scheduler, and garbage collection data.

Using tools like go tool trace or gotraceui can help you inspect these traces and identify issues. Traces are particularly useful for debugging latency problems and can be caught in the act with the help of a flight recorder.

Turn on CPU profiling when taking a trace to see execution with greater detail. If you notice CPU profiling sample events appearing at a rate that doesn't match the sample rate, it might indicate that the OS or platform is taking away CPU time from the process.

Check this out: Golang Time since

Adding new instrumentation with the tracer can be helpful, especially if you want to see events in relation to other scheduling events. However, consider using debuglog for additional instrumentation first, as it's far easier to get started with.

The go tool trace records runtime information and provides a visual web page. It can help you visualize the scheduling flow, including goroutine timelines and processor information.

Related reading: Why Are Schedules Important

Goroutines have three high-level states: Waiting, Runnable, and Executing. These states dictate the role the Go scheduler takes with any given Goroutine.

A Goroutine can be in a Waiting state, stopped and waiting for something to continue. This could be due to system calls or synchronization calls, which are a root cause for bad performance.

A Goroutine is Runnable when it's ready to be executed and waiting for the kernel thread to execute it, or after it has been unblocked.

A Goroutine can transition between these states: from Waiting to Runnable, from Runnable to Running, and from Blocked to Runnable. This transition is managed by the Go scheduler.

Here are the possible states of a Goroutine:

A Goroutine can be in one of three states: Waiting, Runnable, or Executing. These states dictate the role the Go scheduler takes with any given Goroutine.

Waiting means the Goroutine is stopped and waiting for something to continue. This could be for reasons like waiting for the operating system or synchronization calls.

Runnable means the Goroutine wants time on an M so it can execute its assigned instructions. If you have a lot of Goroutines that want time, then Goroutines have to wait longer to get time.

Executing means the Goroutine has been placed on an M and is executing its instructions. This is what everyone wants.

A Goroutine can be in one of the following states:

The deprecated Goroutine scheduler was a major overhaul in Go's concurrency model. It was redesigned in 2012 due to performance issues with the previous implementation, which was abandoned after just four years of use.

The old scheduler worked on the M:N model, where all coroutines (Goroutines) were placed in a global Goroutine queue. This queue was protected by a mutex lock to ensure mutual exclusion and synchronization.

Creating, destroying, and scheduling Goroutines required each M (thread) to acquire the lock, leading to intense lock contention. This resulted in delays and additional system overhead.

Transferring Goroutines between M's also led to delays and poor locality, as related Goroutines were executed on different threads rather than the same one.

System calls resulted in frequent thread blocking and unblocking operations, increasing system overhead.

Goroutines are incredibly lightweight, occupying only a few kilobytes each, which allows Go to support a large number of them within limited memory space, facilitating greater concurrency.

In Go, these coroutines are called Goroutines, and they are derived from the concept of coroutines, allowing a set of reusable functions to run on a set of threads.

Programmers are shielded from the low-level details of Goroutines, which simplifies programming and offers easier concurrency.

A Goroutine's stack may only consume a few kilobytes, but it is dynamically scalable, with the runtime automatically allocating more memory when needed.

The runtime scheduler can still move other Goroutines of the same thread to other runnable threads even if one Goroutine is blocked.

The Go language utilizes Goroutines and Channels to provide a more user-friendly approach to concurrency.

Goroutines are incredibly flexible, with their scheduling handled by the runtime, and their minimal memory usage makes them a great choice for concurrent programming.

The new scheduler design introduced by Go includes a new component called P (processor), in addition to M (threads) and G (coroutines).

A key characteristic of Goroutines is their minimal memory usage, with each Goroutine occupying only a few kilobytes.

In the new scheduler, the global G queue still exists but with weakened functionality, allowing an M to retrieve Goroutines from the global G queue when it cannot find any in the queues of other P's.

Goroutines can be in one of three states: Runnable, Running, or Blocked. A goroutine can be in a runnable state when it is created or after it has been unblocked.

When a goroutine performs a system call, it may take a long time to return, and the thread is essentially sleeping. This can lead to a situation where all the threads assigned to our program are just waiting for system calls to finish.

To solve this problem, we can create a new thread just before a system call. Since we are the ones writing the language, and writing the function to open files, we can just add a simple line before we open the file to create a new thread.

However, creating a new thread just before a system call can lead to a different problem. We would be allocating memory to each thread, which can be problematic for the system.

System calls can be either synchronous or asynchronous. Synchronous system calls can block the M, while asynchronous system calls can be processed by the network poller.

Here are the states of a goroutine:

In the case of a synchronous system call, the scheduler detaches the M from the P with the blocking goroutine and brings in a new M to service the P. This allows the goroutines to continue executing without being blocked.

Goroutines can also be in a long-term sleep state, where they are waiting for an available P to bind to. If there are no idle Ps in the queue, the goroutine will be marked as runnable and added to the global queue.