How to deploy Metabase 0.46.5 in docker swarm behind Caddy v2.6.4

This post is to show you how to deploy Metabase 0.46.5 to our Docker Swarm Cluster using the Docker Compose tool.

Metabase is the easy, open source way for everyone in your company to ask questions and learn from data.

learn more about using the official Metabase website and GitHub.

Let’s start with actual deployment…

Prerequisites

Please make sure you should fulfill the below requirements before proceeding to the actual deployment.

1. Docker Swarm Cluster with GlusterFS as persistent tool.

2. Caddy as reverse proxy to expose micro-services to external.

3. Database stack to host application databases.

Introduction

If you are looking for a tool to easily summarize and visualize your data without ever writing a single line of SQL Query or having to wait on a coworker. Metabase is the perfect tool.

When you need to dig into the complicated stuff, Metabase provides a flexible GUI query builder and an elegant SQL interface.

You can get beautiful graphs and charts with just a few clicks from the data.

Why is Metabase

Metabase is a simple and powerful analytics tool that lets anyone learn and make decisions from their company’s data. No technical knowledge is required! I hope you love it.

Metabase allows us to visualize our data using beautiful GUI. You can browse or search through all tables in our databases, then filter things down to find just what you need.

Schedule and send charts or results to your team via email or Slack.

Let everyone on your team create, organize, and share beautiful collections of visualizations and data.

Set up alerts to let everyone know when something needs your attention, or when you’ve met that goal at last.

Metabase Features

Metabase is the easy, open-source way for everyone in your company to ask questions and learn from data.

• You or anyone on your team ask questions without knowing SQL

• Create rich beautiful dashboards with auto refresh and full screen

• SQL Mode for analysts and data pros

• Create canonical segments and metrics for every one to use in your team

• You can send data to Slack or email on a schedule with Pulses

• Allows us to view data in Slack anytime with MetaBot

• Humanize data for your team by renaming, annotating and hiding fields

• You can see changes in your data with alerts

Metabase Supported Databases

Metabase supports most of the databases. Below is the list of supported databases.

• PostgreSQL

• MySQL

• Druid

• SQL Server

• Redshift

• MongoDB

• Google BigQuery

• SQLite

• H2

• Oracle

• Vertica

• Presto

• Snowflake

• SparkSQL

Persist Metabase Data

Containers are fast to deploy and make efficient use of system resources. Developers get application portability and programmable image management and the operations team gets standard run time units of deployment and management.

With all the known benefits of containers, there is one common misperception that the containers are ephemeral, which means if we restart the container or in case of any issues with it, we lose all the data for that particular container. They are only good for stateless micro-service applications and that it’s not possible to containerize stateful applications.

I am going to use GlusterFS to overcome the ephemeral behavior of Containers.

I already set up a Replicated GlusterFS Volume to have data replicated throughout the cluster if I would like to have some persistent data.

The below diagram explains how the replicated volume works.

Volume will be mounted on all the nodes, and when a file is written to the /mnt partition, data will be replicated to all the nodes in the Cluster

In case of any one of the nodes fails, the application automatically starts on other node without loosing any data and that’s the beauty of the replicated volume.

Persistent application state or data needs to survive application restarts and outages. We are storing the data or state in GlusterFS and had periodic backups performed on it.

We will use a backup of the volume to spin a new application container anywhere else in case of unexpected issues occur in the current environment.

I am going to persistent /metabase-data/metabase.db of Metabase for disorder recovery.

Create folders in /mnt directory to persistent Metabase data folder.

cd /mnt

sudo mkdir -p metabase

Please watch the below video for Glusterfs Installation

https://youtu.be/NUcDn3MxJ1c

Deploy Mariadb in Docker Swarm

Prepare Metabase Environment

Create a folder in /opt directory to place configuration file, i.e, .yml file for Metabase.

Use the below commands to create the folder.

cd /opt

sudo mkdir -p metabase

cd metabase

sudo touch metabase

Metabase Docker Compose

Open metabase.yml created earlier with nano editor using sudo nano metabase.yml

Copy and paste the below code in metabase.yml

Here is the docker compose file for metabase.

version: "3.7"
 
services:
  metabase:
    image: metabase/metabase
    depends_on:
      - maria
    ports:
      - '4000:3000'
    volumes:
      - /mnt/metabase:/metabase-data
    environment:
      - MB_DB_TYPE=mysql
      - MB_DB_HOST=maria
      - MB_DB_PORT=3306
      - MB_DB_DBNAME_FILE=/run/secrets/mysql_db      
      - MB_DB_USER_FILE=/run/secrets/mysql_user
      - MB_DB_PASS_FILE=/run/secrets/mysql_password
      - MB_DB_FILE=/metabase-data/metabase.db
    secrets:
      - mysql_db
      - mysql_user
      - mysql_password
    networks:
      - proxy
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
secrets:
  mysql_db:
    file: ./mysql_db.txt
  mysql_user:
    file: ./mysql_user.txt
  mysql_password:
    file: ./mysql_password.txt
volumes:
  metabase:
    driver: "local"
networks:
  caddy:
    external: true

As I mentioned in the prerequisites, I used MariaDB as a back-end storage system for Wiki.js which was deployed earlier to our Docker Swarm environment.

Caddyfile – Metabase

The Caddyfile is a convenient Caddy configuration format for humans.

Caddyfile is easy to write, easy to understand, and expressive enough for most use cases.

Please find Production-ready Caddyfile for Metabase.

Learn more about Caddyfile here to get familiar with it.

{
    email you@example.com
    default_sni elastic
    cert_issuer acme
    # Production acme directory
    acme_ca https://acme-v02.api.letsencrypt.org/directory
    # Staging acme directory
    #acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
    servers {
        metrics
        protocols h1 h2c h3
        strict_sni_host on
        trusted_proxies cloudflare {
            interval 12h
            timeout 15s
        }
    }
}
metabase.example.com {
    log {
        output file /var/log/caddy/metabase.log {
        roll_size 20mb
        roll_keep 2
        roll_keep_for 6h
        }
        format console
        level error
    }
    encode gzip zstd
    reverse_proxy metabase:3000
}

Please go to Caddy Post to get more insight to deploy it in the docker swarm cluster.

Final Metabase Docker Compose (Including caddy server configuration)

Please find the full docker-compose file below.

If you are not familiar with Caddy Reverse proxy, I already wrote an article Caddy in Docker Swarm. Please go through if you want to learn more.

version: "3.7"

services:
  caddy:
    image: tuneitme/caddy
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 443
        published: 443
        mode: host
        protocol: udp
    networks:
      - caddy
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - /mnt/caddydata:/data
      - /mnt/caddyconfig:/config
      - /mnt/caddylogs:/var/log/caddy
    deploy:
      placement:
        constraints:
          - node.role == manager
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
  metabase:
    image: metabase/metabase
    depends_on:
      - maria
    ports:
      - '4000:3000'
    volumes:
      - /mnt/metabase:/metabase-data
    environment:
      - MB_DB_TYPE=mysql
      - MB_DB_HOST=maria
      - MB_DB_PORT=3306
      - MB_DB_DBNAME_FILE=/run/secrets/mysql_db      
      - MB_DB_USER_FILE=/run/secrets/mysql_user
      - MB_DB_PASS_FILE=/run/secrets/mysql_password
      - MB_DB_FILE=/metabase-data/metabase.db
    secrets:
      - mysql_db
      - mysql_user
      - mysql_password
    networks:
      - proxy
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
secrets:
  mysql_db:
    file: ./mysql_db.txt
  mysql_user:
    file: ./mysql_user.txt
  mysql_password:
    file: ./mysql_password.txt
volumes:
  caddydata:
    driver: "local"
  caddyconfig:
    driver: "local"
  caddylogs:
    driver: "local"   
networks:
  caddy:
    external: true

Here I used a custom Caddy docker container with plugins, like Cloudflare DNS, Caddy Auth Portal etc…

Please find the custom caddy docker image below.

Tuneit Caddy Docker Image

Deploy Metabase using Docker Compose

Please make sure we have created a wiki database before deploying Metabase using the MariaDB stack deployed earlier.

Now it’s time to deploy our docker-compose file above metabase.yml using the below command

docker stack deploy --compose-file metabase.yml metabase

You can give it any name for the stack. I just named it as metabase

Check the status of the stack by using docker stack ps metabase

Now open any browser and type metabase.example.com (whatever host URL used in the Metabase configuration in the docker-compose file) to complete Metabase installation.

Make sure that you have DNS entry for your application (metabase.example.com) in your DNS Management Application.

Please find below images for your reference.

Hope you enjoyed this tutorial, please give your input/thought on it by commenting below. It would help me to bring more articles that focus on Open Source to self-host.

Stay tuned for other deployments in coming posts… 🙄