Building PgPulse as a Native PostgreSQL Extension

A couple of months ago, I built PgPulse, a standalone Rust binary that connects to PostgreSQL instances, monitors the health of primary and replica servers, evaluates replication lag, and exports everything as Prometheus metrics.

It did exactly what I wanted it to do.

If you're interested in that version, you can read about it here: Building PgPulse

But after using it for a while, something kept bothering me.

Not the monitoring. Not the exporter. The deployment.

Every PostgreSQL cluster now needs one more binary running somewhere. Another service to build, deploy, configure, and monitor. It wasn't exactly painful, but it felt… unnecessary.

PostgreSQL already has the ability to run code inside the server. So naturally, the next question became:

What if PgPulse itself became a PostgreSQL extension?

It sounded like a fun weekend project. It absolutely wasn't.

The Idea

Instead of having an external process connect to every PostgreSQL instance, why not move all of the collection logic inside PostgreSQL itself?

The extension would:

• collect metrics
• connect to replica instances
• evaluate the health
• store the latest state internally

Then an exporter becomes almost boring. It simply connects to the primary database, reads whatever PgPulse has already computed, and exposes it to Prometheus (or anything else).

Replica
   │
   ▼
┌──────────────────────────────┐
│ PostgreSQL Primary           │
│                              │
│ PgPulse Extension            │
│ ├── Background Worker        │
│ ├── SPI Queries              │
│ ├── Replication Collector    │
│ ├── Health Evaluator         │
│ ├── Shared Memory            │
│ └── SQL Views                │
└──────────────┬───────────────┘
               │
               ▼
        Thin Rust Exporter
               │
               ▼
          Prometheus

Simple enough. At least that's what I thought.

This Wasn't the Original Plan

One thing worth mentioning is that this wasn't the architecture I started with. Not even close.

I rewrote parts of this project multiple times. Some ideas looked fantastic on paper. Some worked until they didn't. Some were simply me not understanding PostgreSQL well enough.

The architecture above is simply where I eventually landed after enough iterations.

Welcome to PostgreSQL Internals

Until this project, PostgreSQL was just… PostgreSQL. I connected to it. Ran SQL. Did CRUD. Built APIs around it. Never really cared about what happened underneath.

Building an extension changes that perspective completely.

You're no longer talking to PostgreSQL. You're now running inside PostgreSQL.

That meant learning a completely different side of the database.

Background Workers

The first thing I came across was Background Workers. Think of them as PostgreSQL-managed processes.

Instead of creating another daemon that runs beside PostgreSQL, you register a worker and let the Postmaster manage its lifecycle.

• If PostgreSQL crashes… the worker crashes.
• When PostgreSQL comes back… the worker comes back.

Exactly what I needed.

The background worker eventually became the heart of PgPulse. Every few seconds it wakes up, collects metrics, evaluates the cluster health, stores everything, and goes back to sleep. Repeat forever.

SPI

Next came SPI, the Server Programming Interface.

Normally, we query PostgreSQL from an application. SPI lets an extension do the exact opposite. You're already inside PostgreSQL, but sometimes you still want to execute SQL. SPI gives you that bridge.

Most of PgPulse's internal metrics are collected using SPI queries.

GUC

Every extension eventually needs configuration:

• Replica addresses
• Polling intervals
• Thresholds
• Timeouts

Instead of inventing another configuration format, PostgreSQL already has one: GUC (Grand Unified Configuration).

So all runtime configuration now lives there. It feels much more natural because users configure PgPulse exactly the same way they configure PostgreSQL itself.

Shared Memory

This deserves an entire section because this one nearly broke me.

Initially, I thought shared memory would just be another struct. Turns out PostgreSQL has opinions. Very strong opinions.

One thing I learnt (after way too many hours of debugging) is that heap-backed Rust types simply don't belong in PostgreSQL shared memory. Things like:

• Vec
• String
• HashMap

They're all heap allocated. Shared memory needs data structures that are completely self-contained.

Guess what most of my models were using? Exactly.

So I had to go back and redesign a good chunk of the project. I ended up replacing those structures with fixed-capacity equivalents using the heapless crate.

The annoying part? Everything built successfully. Every unit test passed. The extension loaded.

And then… nothing.

Sometimes the worker silently crashed. Sometimes PostgreSQL happily reported that the extension was loaded while absolutely nothing happened.

That single mistake probably cost me more time than writing half the extension.

Lightweight Locks

Once multiple PostgreSQL processes start touching shared memory, synchronisation becomes important.

PostgreSQL has its own synchronisation primitive called LWLocks (Lightweight Locks).

• The background worker acquires a write lock before updating metrics.
• Readers acquire a read lock before exposing them.

Simple enough. Thankfully pgrx wraps most of this quite nicely.

pgrx

The project is written entirely in Rust using pgrx. Honestly, it does an incredible amount of heavy lifting.

The documentation, however… let's just say I spent far more time reading example projects than reading the actual documentation.

The APIs you'll quickly become friends with are:

• pg_module_magic
• #[pg_extern]
• #[pg_guard]
• _PG_init
• PgLwLock
• extern "C-unwind"

Once those start making sense, PostgreSQL extensions stop feeling quite so magical.

Organising the Project

After a couple of rewrites, the project eventually settled into something like this:

src/
├── collectors/
│   ├── queries.rs
│   └── replication.rs
├── evaluator.rs
├── guc.rs
├── models.rs
├── shared_memory.rs
├── worker.rs
└── lib.rs

exporter/

Each module ended up having a fairly well-defined responsibility.

guc.rs — Stores every runtime configuration.

models.rs — Contains PostgreSQL-compatible data structures. This was also where I had to replace every heap-backed type.

collectors/ — One collector queries PostgreSQL through SPI. The other connects to replica instances.

evaluator.rs — Takes all the collected metrics and computes the overall cluster health. Instead of exporting dozens of raw numbers and expecting Prometheus to figure everything out, the extension already knows whether the cluster is healthy or not.

shared_memory.rs — Stores the latest state. Probably the only place in the project where I had to touch unsafe. Fortunately, the unsafe surface stayed fairly small.

worker.rs — The actual monitoring loop:

1. Wake up.
2. Read configuration.
3. Collect metrics.
4. Connect to replicas.
5. Evaluate health.
6. Write into shared memory.
7. Sleep.
8. Repeat.

Installing the Extension

Building a PostgreSQL extension is… different. Unlike a normal Rust binary, cargo build it isn't the finish line.

1. The compiled .so file has to be copied into PostgreSQL's library directory.
2. The SQL file defining the extension has to be copied into PostgreSQL's extension directory.
3. PostgreSQL then needs to preload the extension.
4. Restart.
5. Create the extension.

Only then do your views appear inside the database.

Debugging Extensions Is an Experience

This was easily the hardest part of the project. Your development loop looks something like this:

cargo pgrx build
       ↓
   Copy .so
       ↓
   Copy SQL
       ↓
Restart PostgreSQL
       ↓
 Reload extension
       ↓
  Cross fingers.

Sometimes PostgreSQL tells you everything loaded successfully. Except your worker never started.

• Was _PG_init() called?
• Did the Postmaster register the worker?
• Did the worker panic before entering the loop?
• Did PostgreSQL silently reject something?

Good luck.

I spent an embarrassing amount of time staring at logs wondering why nothing was happening. The answer, more often than not, was still that one innocent-looking Vec sitting inside shared memory.

One More Thing

Extensions are tightly coupled to PostgreSQL versions. If you compile against PostgreSQL 16 and try loading it into PostgreSQL 17… you're going to have a bad time.

Always compile against the exact PostgreSQL version you're deploying. You'll save yourself a lot of unnecessary debugging.

The Exporter Became… Boring

Ironically, the exporter used to be the entire project. Now it's probably the simplest component.

It's just a Tokio service:

1. Connect to one database.
2. Read the PgPulse views.
3. Expose Prometheus metrics.
4. Done.

Almost all the complexity now lives where it probably should have lived from the beginning—inside PostgreSQL.

Final Thoughts

I started this project because I wanted to remove one more binary from deployment. I ended up learning far more about PostgreSQL than I ever expected:

• Background Workers
• SPI
• GUCs
• LWLocks
• Shared memory
• And the many creative ways a single Vec can ruin your weekend

I'm still nowhere near an expert in PostgreSQL internals, and I'm sure there are cleaner ways of implementing parts of PgPulse. But I guess that's the fun part. Building something is probably the fastest way to discover how much you don't know.

If you're interested in the code, it's all available on GitHub: HarshitRuwali/PgPulse

And if this blog saves someone else from spending two days debugging a Vec inside shared memory… I'd call that a success.