Using Vmstats and Statix in Elixir

When my employer Traitify started using Elixir, we did what good software devs do and used the top hits on Github or Google whenever we needed a library for something. Need to put something in S3? Well looks like ex_aws is the best one! This is how we ended up with ex_statsd and ex_vmstats in our projects.

When you check out ex_statsd now, you’ll see that it’s no longer maintained. However, we discovered that all may not be well with it during load testing when we saw tens of thousands of messages in the ExStatsd process msg queue. Fortunately for the community, statix is an excellent alternative that we’re using now.

Unfortunately ex_vmstats only works with ex_statsd. But I had become familiar with using Erlang libraries in Elixir projects and so gave a crack at integrating the original Erlang vmstats library with statix.

First things first, setup a statix socket module following the directions on it’s Github repo. Luckily for us, vmstats exists in hex.pm so adding it our Elixir projects is as easy as adding {:vmstats, "~> 2.2"} to our mix.exs.

Next, we’ll need to implement the :vmstats_sink behaviour in a module like so.

defmodule MyApp.Stats.StatixVmstatsSink do
  @moduledoc false
  #This is our Statix socket module
  alias MyApp.Stats.Statix

  @behaviour :vmstats_sink

  def collect(:counter, key, value) do
    Statix.set(key, value)
  end

  def collect(:gauge, key, value) do
    Statix.gauge(key, value)
  end

  def collect(:timing, key, value) do
    Statix.timing(key, value)
  end
end

The last step is to update our config.exs to tell vmstats to use our sink.

config :vmstats,
  sink: Profiles.Stats.StatixVmstatsSink,
  #this interval is in milliseconds
  interval: 3000

And that should do it! Your vmstats will show up in your statsd server. One of the first things that I noticed in our stats was the number of VM Reductions at idle.

Before:

graph showing 1 million plus reductons per a box at idle

And after: graph showing 2-5 reductions per a box at idle

The number of total Reductions per a machine went down from over 1 million to just a handful! If you don’t know what Reductions are, they influence the VM preemptive scheduler, and in general you want to minimize them. You can read more about them here.

Thanks for reading, and remember that Erlang has many great libraries out there, don’t be afraid to use them!