The SSL Certificate Saga: Local DNS, Nginx and My Homelab

Project Goal

Configure a secure local environment with: - Public domain olyhome.site (purchased from Hostinger) - Local DNS server (Bind9) to manage internal subdomains - Nginx Proxy Manager (NPM) for reverse proxy - HTTPS-secured local subdomains (e.g., proxmox.olyhome.site, n8n.olyhome.site) - Without exposing services to the internet

Architecture Overview

Local Network
    ↓
Bind9 (Local DNS)
├── proxmox.olyhome.site → 192.168.11.100
├── n8n.olyhome.site → 192.168.11.100
└── db.olyhome.site → 192.168.11.100
    ↓
Nginx Proxy Manager (192.168.11.100)
├── proxmox.olyhome.site → https://192.168.11.254:8006
├── n8n.olyhome.site → http://192.168.1.20:5678
└── db.olyhome.site → http://192.168.1.30:5432

Steps Completed

Step 1: Bind9 Configuration (Local DNS)

Why Bind9?

Use Case: - Control DNS resolution for local domain - Override public DNS with local IPs - Manage internal subdomain routing - No reliance on external DNS providers

Installation

# Install Bind9
sudo apt update
sudo apt install bind9 bind9-utils -y

# Create zone directory
sudo mkdir -p /etc/bind/zones

Zone Configuration

# Edit named.conf.local
sudo nano /etc/bind/named.conf.local
// Zone for olyhome.site
zone "olyhome.site" {
    type master;
    file "/etc/bind/zones/db.olyhome.site";
};

Zone File Creation

# Create zone file
sudo nano /etc/bind/zones/db.olyhome.site
$TTL    604800
@       IN      SOA     ns1.olyhome.site. admin.olyhome.site. (
                        2025102601      ; Serial
                        604800          ; Refresh
                        86400           ; Retry
                        2419200         ; Expire
                        604800 )        ; Negative Cache TTL
;
@       IN      NS      ns1.olyhome.site.
ns1     IN      A       192.168.11.100

; Proxy server (Nginx Proxy Manager)
@       IN      A       192.168.11.100

; Service subdomains
proxmox IN      A       192.168.11.100
n8n     IN      A       192.168.11.100
db      IN      A       192.168.11.100
nginx   IN      A       192.168.11.100

Restart and Test

# Check configuration
sudo named-checkconf
sudo named-checkzone olyhome.site /etc/bind/zones/db.olyhome.site

# Restart Bind9
sudo systemctl restart bind9

# Test DNS resolution
dig proxmox.olyhome.site @localhost

# Expected: Answer section shows 192.168.11.100

Step 2: Deploy Bind9 in Docker

Why Docker?

  • Isolation from host system
  • Easy backup and restoration
  • Portable configuration
  • No conflict with system resolver

Docker Compose Configuration

version: '3.8'

services:
  bind9:
    image: ubuntu/bind9:latest
    container_name: bind9-dns
    restart: unless-stopped
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "953:953/tcp"  # rndc
    volumes:
      - ./config:/etc/bind
      - ./zones:/var/lib/bind
      - ./cache:/var/cache/bind
    environment:
      - TZ=Europe/Paris
      - BIND9_USER=bind
    networks:
      - dns_network

networks:
  dns_network:
    driver: bridge

Conflict Resolution

Problem: Port 53 already in use by systemd-resolved

# Check what's using port 53
sudo ss -tlnp | grep :53

# Disable systemd-resolved
sudo systemctl disable systemd-resolved
sudo systemctl stop systemd-resolved

# Remove resolv.conf symlink
sudo rm /etc/resolv.conf

# Create static resolv.conf
echo "nameserver 127.0.0.1" | sudo tee /etc/resolv.conf

# Start Bind9 container
docker-compose up -d

# Verify port 53 is bound
docker logs bind9-dns

Step 3: Nginx Proxy Manager Configuration

Deployment

version: '3'
services:
  nginx-proxy-manager:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginx-proxy-manager
    restart: unless-stopped
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    networks:
      - proxy_network

networks:
  proxy_network:
    driver: bridge
# Deploy NPM
docker-compose up -d

# Access web interface
http://192.168.11.100:81

# Default credentials:
# Email: admin@example.com
# Password: changeme

Create Proxy Hosts

For Proxmox:

Domain: proxmox.olyhome.site
Scheme: https
Forward Hostname/IP: 192.168.11.254
Forward Port: 8006
Websockets: ✓
Block Common Exploits: ✓

For n8n:

Domain: n8n.olyhome.site
Scheme: http
Forward Hostname/IP: 192.168.1.20
Forward Port: 5678
Websockets: ✓ (CRITICAL for n8n)

Step 4: SSL Certificate Attempt

Goal: Valid SSL Certificates for Local Domains

Challenge: Nginx Proxy Manager doesn’t support Hostinger for automated DNS challenges.

Method: Manual Certbot with DNS Challenge

# Install certbot
sudo apt install certbot -y

# Request certificate manually
sudo certbot certonly \
  --manual \
  --preferred-challenges dns \
  -d olyhome.site \
  -d '*.olyhome.site'

Process: 1. Certbot provides TXT record value 2. Add TXT record to Hostinger DNS: _acme-challenge.olyhome.site TXT "generated-value-here" 3. Wait for DNS propagation (2-5 minutes) 4. Press Enter in certbot 5. Certificate generated successfully

Certificate Location:

/etc/letsencrypt/live/olyhome.site/fullchain.pem
/etc/letsencrypt/live/olyhome.site/privkey.pem

Step 5: NPM Import Issue

Problem Encountered

Error: NPM refuses to import privkey.pem

Message: “Private key is encrypted/has passphrase”

Attempted Solutions

Solution 1: Remove Passphrase with OpenSSL RSA

# Try to remove passphrase
openssl rsa -in privkey.pem -out privkey-nopass.pem

# ERROR: "Not an RSA key"
# Let's Encrypt uses ECDSA keys, not RSA!

Solution 2: Convert EC Key

# Check key type
openssl pkey -in privkey.pem -text -noout

# Convert EC key (remove passphrase if any)
openssl ec -in privkey.pem -out privkey-nopass.pem

# Success: privkey-nopass.pem created

Solution 3: Import into NPM

NPM UI:
SSL Certificates → Add SSL Certificate → Custom
Certificate Key: Upload privkey-nopass.pem
Certificate: Upload fullchain.pem
Intermediate Certificates: (leave empty)

Result: Import still fails with generic error

Current Problem

Status: Unable to import valid Let’s Encrypt certificate into Nginx Proxy Manager despite removing any encryption from private key.

Error: NPM shows generic “Failed to save certificate” message without specific details.

Troubleshooting Performed

1. Verify Certificate Format

# Check fullchain.pem
openssl x509 -in fullchain.pem -text -noout

# Verify it contains:
# - Certificate
# - Subject Alternative Names (SAN): *.olyhome.site
# - Issuer: Let's Encrypt Authority

# Check privkey.pem
openssl pkey -in privkey-nopass.pem -check

# Output: "EC Key valid"

2. Check for Extra Characters

# Ensure no extra whitespace/characters
head -n 1 fullchain.pem
# Should be exactly: -----BEGIN CERTIFICATE-----

tail -n 1 fullchain.pem
# Should be exactly: -----END CERTIFICATE-----

3. Try Different Import Methods

Method A: Via NPM UI ❌ Failed

Method B: Direct File Copy

# Copy to NPM container
docker cp fullchain.pem nginx-proxy-manager:/data/custom_ssl/
docker cp privkey-nopass.pem nginx-proxy-manager:/data/custom_ssl/

# Try importing via UI using container paths
# Still fails

Method C: Manual Nginx Configuration

# Access NPM container
docker exec -it nginx-proxy-manager bash

# Edit proxy host config directly
nano /data/nginx/proxy_host/1.conf
server {
    listen 443 ssl http2;
    server_name proxmox.olyhome.site;

    ssl_certificate /data/custom_ssl/fullchain.pem;
    ssl_certificate_key /data/custom_ssl/privkey-nopass.pem;

    location / {
        proxy_pass https://192.168.11.254:8006;
        # ... rest of proxy config
    }
}

Alternative Solutions Considered

Option 1: Use Self-Signed Certificate

# Generate self-signed cert in NPM format
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout selfsigned-key.pem \
  -out selfsigned-cert.pem \
  -subj "/CN=*.olyhome.site"

# Import into NPM (works but browser warnings)

Option 2: Use mkcert

# Install mkcert
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
sudo mv mkcert-v1.4.4-linux-amd64 /usr/local/bin/mkcert
sudo chmod +x /usr/local/bin/mkcert

# Install local CA
mkcert -install

# Generate certificate
mkcert "*.olyhome.site" olyhome.site

# Import into NPM (should work, client CA installation required)

Option 3: Manual Nginx Reverse Proxy

# Skip NPM entirely
# Install nginx directly
sudo apt install nginx -y

# Configure virtual hosts manually
# More control, no GUI restrictions

Option 4: Caddy Server

# Install Caddy
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg

# Caddy automatically handles Let's Encrypt
# Simple config:
proxmox.olyhome.site {
    reverse_proxy https://192.168.11.254:8006 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

Lessons Learned

1. NPM Limitations

  • Doesn’t support all DNS providers for automated challenges
  • Custom certificate import can be finicky
  • Limited error reporting for troubleshooting
  • Alternative reverse proxies may be more flexible

2. Certificate Format Matters

  • Let’s Encrypt uses ECDSA keys (not RSA)
  • Use openssl ec for ECDSA, not openssl rsa
  • PEM format must be exact (no extra whitespace)

3. Local DNS with Public Domain Works

  • Bind9 successfully overrides public DNS
  • Clients resolve internal IPs for public domain
  • Enables HTTPS for local services with valid certs

4. DNS Challenge is Powerful

  • Works without opening ports 80/443
  • Enables wildcard certificates
  • Requires manual DNS record updates for providers not supported by certbot plugins

Next Steps

Short Term

  1. Test mkcert solution:
    • Install mkcert
    • Generate certificates
    • Import into NPM
    • Install CA on client devices
  2. Try Caddy as alternative:
    • Automatic HTTPS
    • Simpler configuration
    • Better error reporting

Long Term

  1. Automate certificate renewal:
    • Create script to update DNS TXT record
    • Run certbot renewal monthly
    • Auto-import into NPM (if possible)
  2. Consider Traefik:
    • Docker labels for configuration
    • Automatic Let’s Encrypt integration
    • Better suited for containerized environments
  3. Implement monitoring:
    • Certificate expiry alerts
    • DNS resolution monitoring
    • Proxy health checks

Files Generated

/etc/letsencrypt/live/olyhome.site/
├── fullchain.pem        # Complete certificate chain
├── privkey.pem          # Original private key (ECDSA)
├── cert.pem             # Certificate only
└── chain.pem            # Intermediate certificates

/home/f4blox/Desktop/
└── privkey-nopass.pem   # Private key without passphrase

Conclusion

While the SSL certificate import into NPM remains unresolved, the project successfully demonstrated: - Local DNS configuration with Bind9 - Split-horizon DNS (internal IPs for public domain) - Nginx Proxy Manager proxy host configuration - Manual Let’s Encrypt certificate generation with DNS challenge

The inability to import certificates into NPM highlights the value of exploring alternative reverse proxy solutions like Caddy or Traefik, which offer better automation and fewer import restrictions.

Key Takeaway: Sometimes the tool limitation isn’t a failure—it’s an opportunity to discover better solutions.

Next article will explore Caddy deployment as NPM alternative.