Go at refurbed

Refurbed is a Golang shop since the beginning. Our founder and CTO built the first version of the platform using Go. He had successfully built services with Go in earlier challenges and decided to use it again.

The first commit in our repository was June, 2017. Go 1.8 had been released some months prior. Since then, Go has grown massively and became the default “cloud” language and one of the most popular backend languages.

Why refurbed uses Go

Go scales well with Engineering teams, and refurbed is a growing organization.

The simplicity of the language makes it easy to pickup. Being statically typed scales well in big code-bases. The light syntax, garbage collection and convention over configuration keeps it as productive as a scripting language.

The high quality of the tooling allows developers to develop on all popular platforms without extra dependencies. It is self-contained. The scope of the tooling, including testing, embedding, cross-compilation, formatting reduces the choices organizations have to make, and ensures the quality.

Go is well designed and aiming for production systems. Features are introduced in stages and there is an emphasis on boring and stable over breaking backwards compatibility.

The standard library extends beyond the operating system other system languages define as the core library. It includes elements used in backend and distributed systems: HTTP, JSON.

This way one can avoid 3rd party libraries. When this is not possible, those can use the standard interfaces provided by the standard library. This reduces the number of choices to make and avoids the framework fatigue that plagues other platforms.

How refurbed uses Go

We embrace Cloud native with a brush of UNIX tradition and simplicity. We do have multiple services, but keep things simple and easy to reason about.

Our platform is a monorepo from where a handful of services (refbwebd, refbapid, refbworkerd, …) are built. Those are then containerized and deployed to Kubernetes.

We do use 3rd party libraries, but we try to keep those to the minimum and they consist of utility libraries (go-env, goqu, kingpin, zerolog, etc) and clients (go-redis, opentelemetry, prometheus, etc) over frameworks. An exception is gin.

API and domains

Since we moved to a Domain Driven Design approach to scale teams. Our business logic is a collection of domains, each one implementing its repositories.

For example, the order domain looks something like this:

domain/order/postgres/filters.go
domain/order/postgres/refunds.go
domain/order/postgres/repository.go
domain/order/postgres/types.go
domain/order/errors.go
domain/order/service.go
domain/order/types.go

A typical service would define a Repository interface, and then offer a database implementation for that repository. We use goqu SQL builder library to access the database.

This allows injecting the repository implementation:

type Service struct {
	Repository        Repository
	// ...
}

The service then exposes the business methods for that domain:

func (s Service) SearchOrdersForMerchant(ctx context.Context, merchantID int64, q Query) (SearchResult, error)

The domains are exposed in a few API servers (like refbapid) using gRPC. APIs are grouped by the consumer (merchant, customer, etc). APIs are logically separated in internal/external for API stability and in a separate service for scalability requirements.

Other logic like authentication and logging is implemented using gRPC interceptors.

Our internal APIs are also consumed from frontends using a grpc-web proxy.

We also have an older REST API server (refbd) we are phasing out, using the Gin framework, and accessing the database using Gorm.

Task Processors

Our worker processor (refbworkerd) is a Go client using RabbitMQ as an event queue.

We have a common framework net/amqp/worker which allows to easily write specialized workers (eg. net/worker/worker_mailer) encapsulating life-cycle and retry logic. refbdworkerd instantiates then all workers.

Web server

Our platform consist of a front web server (refbwebd) written in Go using the Gin framework. This serves a high-performance website optimised for SEO.

HTML meets Go using the quicktemplate engine. Unlike the standard templates, they are converted into Go code and then compiled at the build time.

Templates allow our web developers to build raw HTML without compromising Go’s performance:

{% import "refurbed/web/types" %}

{% code
type SomePage struct {
	Data types.Data
}
%}

{% func (p *SomePage) Body() %}
{% collapsespace %}
  <div class="example">
  <!-- more html -->
  {%= p.Data.SomeField %}
  </div>
{% endfunc %}

This template is compiled with qtc quicktemplate compiler and the writeTemplate function takes care of writing the Body() function call result in the response.

Our handlers are structured by page for example web/handle_product.go. Our refbwebd server provides additional infrastructure to the handlers, like access to the backend APIs or to caching layer (backed by Redis).

A simplified version of our typical web handler for example purposes looks similar to this:

func (s *Server) handleSomePage(c *gin.Context) {
	// bind parameters from handler
	q := struct {
		Param string `form:"param"`
	}{}

	if err := c.ShouldBindQuery(&q); err != nil {
		s.abortWith(c, err)
		return
	}

	s.cached(c).Configure(
		cache.WithTTL(CacheTTLSomePage),
		cache.WithGrace(CacheGraceSomePage))

	data := &types.SomeData{
		...
	}

	errs := make([]error, N)
	wg := sync.WaitGroup{}

	wg.Add(1)
	go func() {
		u := fmt.Sprintf("/v1/api/param/%d/", q.Param)
		errs[0] = s.api(c).Do(GET, u, data)
		wg.Done()
	}()

	// more concurrent API calls
	/// ...

	wg.Wait()

	p := &tpl.SomePage{
		BasePage: tplL.NewBasePage(s.tplOpts(c)),
		SomeData: data,
	}

	//...

	writeTemplate(c, p)
}

The combination of Go, Gin and Quick-templates result in a structured approach for server-side web applications. A typical handler:

Binds parameters from the request
Configures the cache
Makes concurrent API calls and wait for them to complete
Fills templates with data
Renders the templates and set result code

Other cross-cutting concerns like authentication and logging are handled by middlewares and functions wrapping the handlers.

Conclusion

This article has given an overview of how Go is used at refurbed: business logic, APIs, task processing and web pages.

Refurbed has successfully been using Go across the whole platform and services for 5 years and we intend to continue doing so.