Setup Web Server (apache) and DNS (bind) on a MacMini using Homebrew

Contents

  1. Online references
  2. Common pitfalls, problems, useful commands
  3. XCODE and BBEdit - command line tools
  4. BREW
  5. APACHE
  6. HTTPD config edits including HTTP2 and vhosts
  7. HTML default header
  8. Let's Encrypt - certbot
  9. Named and bind
  10. Network Performance tools
  11. SSH on a different port number
  12. PHP
  13. MacMini Setup (for SC/webcam/demo)
  14. TimeMachine Problem Solving
  15. How this page is made

Online references

In this table are the commands for apache, bind and certbot via brew as in https://getgrav.org/blog/macos-mojave-apache-multiple-php-versions.

The previous Apple server software needs to be completely uninstalled!
In this guide the commands are in green. Text to be changed or entered are in light blue.

Common pitfalls, problems, useful commands
  • After a brew upgrade some services may not work. One difficult to diagnose problem may be the firewall settings in MacOSX. It blocks by default incoming connections on unsigned bin's and executables need to be added in the list in the System Preferences Firewall settings. Also note that just putting there the /usr/local/bin/executable will not work since brew puts aliasses (links) there. Follow the link to the real bin. Same for /usr/local/sbin .
  • After a "brew upgrade" the output may display an error that an old directory could not be removed and a sudo command is displayed. In reality the "brew upgrade" did not complete, so execute that sudo command and repeat "brew upgrade" and displayed error - sudo commands until nothing is being done anymore by that command.
  • Somehow launchd has a different or incomplete PATH environment. Caused certbot renew to initially fail.
  • if you need to log out another user:
    • ps awwwwux | grep loginwindow
    • kill the pid of that user with:
    • sudo kill -9 [pid]
  • Support files can be found HERE.

XCODE and BBEdit - command line tools
Apple's command line tools need to be installed, XCode is not needed.
  • xcode-select --install

If needed to re-install command tools do this command first:

  • sudo rm -rf /Library/Developer/CommandLineTools

Install BBEdit from https://www.barebones.com/products/bbedit/index.html

Start BBEdit and set up its command line tools under the BBEdit menu.

BREW
See::
Setup:
  • ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • brew install openldap libiconv
Maintenance, to upgrade all installed programs to newest version:
  • brew doctor
  • brew --version
  • brew update
  • brew upgrade
If needed to correct permissions:
  • sudo chown -R "$USER":admin /usr/local
  • sudo chown -R "$USER":admin /Library/Caches/Homebrew

APACHE
To install Apache:
  • sudo apachectl stop
  • sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
  • brew install httpd
  • sudo brew services start httpd
  • ps -aef | grep httpd
To restart httpd:
  • sudo brew services restart httpd
To debug:
  • tail -f /usr/local/var/log/httpd/error_log sudo apachectl stop
  • sudo apachectl -k restart
  • To get setup paths:
  • sudo apachectl -S
  • Test conf files, must be sudo to test certificates:
  • sudo apachectl configtest

HTTPD config edits including HTTP2 and vhosts
HTTPD config basic edits:
  • bbedit /usr/local/etc/httpd/httpd.conf
  1. Listen 8080 => Listen 80
  2. enable ==> LoadModule deflate_module lib/httpd/modules/mod_deflate.so
  3. enable ==> LoadModule rewrite_module lib/httpd/modules/mod_rewrite.so
  4. ServerAdmin admin@domain.net
  5. #ServerName www.example.com:8080 ==> ServerName localhost:80
  6. enable ==> Include /usr/local/etc/httpd/extra/httpd-autoindex.conf
HTTPD config edits to enable http2
  • bbedit /usr/local/etc/httpd/httpd.conf
  1. disable ===> #LoadModule mpm_prefork_module lib/httpd/modules/mod_mpm_prefork.so
  2. enable ===> LoadModule mpm_event_module lib/httpd/modules/mod_mpm_event.so
  3. enable ===> LoadModule http2_module lib/httpd/modules/mod_http2.so
  4. add ===> Protocols h2 h2c http/1.1
HTTPD config edits enable vhosts:
  • bbedit /usr/local/etc/httpd/httpd.conf
  1. enable ==> LoadModule vhost_alias_module lib/httpd/modules/mod_vhost_alias.so
  2. enable ==> Include /usr/local/etc/httpd/extra/httpd-vhosts.conf
and for vhosts including a catch directory. The catch directory is the default webroot where all requests go to if there is not a specific webroot for that (sub)domain defined. Therefore, it must be the first in the list. Here an example for example.com edit:
  • bbedit /usr/local/etc/httpd/extra/httpd-vhosts.conf
<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
</VirtualHost>
<Directory "/usr/local/var/www">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

<VirtualHost *:80>
    DocumentRoot "/Users/example/Sites"
    ServerName example.com
    ServerAlias www.example.com
</VirtualHost>
<Directory "/Users/example/Sites">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

HTML default header
Web pages header:

<!DOCTYPE html>
<html>
  <head>
    <title>title</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=0.25, maximum-scale=4.0, user-scalable=yes">
    <meta name="description" content="title">
    <meta name="author" content="Cees de Laat">
  </head>
  <body style="font-family: Helvetica,Arial,sans-serif;" text="#ffffff" bgcolor="#000000" link="#ffcc66" alink="#ff9900" vlink="#ffff66">

Let's Encrypt - certbot

  • sudo install -d -o $(whoami) -g admin /usr/local/Frameworks
  • brew install certbot
HTTPD edits:
  • bbedit /usr/local/etc/httpd/httpd.conf

Uncomment:

LoadModule ssl_module modules/mod_ssl.so
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so

add at the end:

<IfModule mod_ssl.c>
   Listen 443
</IfModule>
Include /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf

The idea is that all requests that get through on port 80 or with incorrect url's/domain names end up in:
  • "/usr/local/var/www"
and all requests that come in via https and correct domain names go to the correct webroots of those domains.

For that purpose different rewrite rules take care of redirection. Those are in httpd-vhosts.conf
I use the following domain construction for example.com:
  • catch example.com
    • catches all (sub)domains for which no other webroot is defined is
  • example.com
    • this goes to the normal webroot of the domain
  • www.example.com
    • also to the typical webroot of example.com
  • sub1.example.com
    • a subdomain of example.com with its own webroot
in httpd-vhosts.conf:
  • bbedit /usr/local/etc/httpd/extra/httpd-vhosts.conf
<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
</VirtualHost>
<Directory "/usr/local/var/www">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName example.com
    ServerAlias www.example.com
RewriteEngine on
RewriteCond %{SERVER_NAME} =example.com [OR]
RewriteCond %{SERVER_NAME} =www.example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<Directory "/Users/example/Sites">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>
and create/edit httpd-vhosts-le-ssl.conf:
  • bbedit /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf
<VirtualHost *:443>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
</VirtualHost>

<VirtualHost *:443>
    DocumentRoot "/Users/example/Sites"
    ServerName example.com
    ServerAlias www.example.com
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
</VirtualHost>
To request the certificates:
  • sudo certbot --apache
or if one first wants to do a number of test runs for debugging:
  • sudo certbot --apache --staging
for after successful testing forcing a full new certificate:
  • sudo certbot --apache --force-renewal
for production:
  • sudo certbot renew

if test runs are needed:

  • sudo certbot renew --dry-run
To list the certificates:
  • sudo certbot certificates
See:
Only the plist in the above solution does not work because somehow the PATH variable of the running deamon is not correct. Therefore we make our own script that sets the path and invokes certbot renew, and then we use a LaunchDeamon to periodically invoke the script. For renewal we have to make a shell script and a launchd plist.

create file /usr/local/etc/certbot-renew.sh :
  • bbedit /usr/local/etc/certbot-renew.sh
and put in:

#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
sudo certbot renew

Make it executable:
  • chmod +x /usr/local/etc/certbot-renew.sh

create plist:

  • bbedit /Library/LaunchDaemons/com.letsencrypt.renew.plist
or
  • sudo vi /Library/LaunchDaemons/com.letsencrypt.renew.plist
put in that file the xml below here:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
    <string>com.letsencrypt.renew</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/etc/certbot-renew.sh</string>
    </array>
   <key>StandardErrorPath</key>
    <string>/tmp/com.letsencrypt.renew.stderr</string>
    <key>StandardOutPath</key>
    <string>/tmp/com.letsencrypt.renew.stdout</string>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>4</integer>
        <key>Minute</key>
        <integer>56</integer>
    </dict>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>
  • sudo chmod 644 /Library/LaunchDaemons/com.letsencrypt.renew.plist
  • sudo chown root:admin /Library/LaunchDaemons/com.letsencrypt.renew.plist
  • sudo launchctl load /Library/LaunchDaemons/com.letsencrypt.renew.plist
  • sudo launchctl list | grep -i letsencrypt
  • tail -f -n 40 /tmp/com.letsencrypt.renew.stderr
  • tail -f -n 40 /tmp/com.letsencrypt.renew.stdout
  • sudo tail -f -n 400 /var/log/letsencrypt/letsencrypt.log

Some more info:

The configuration file is at: /etc/letsencrypt/renewal/

Named and bind

  • brew install bind
  • sudo brew services start bind
  • sudo brew services restart bind
  • sudo brew services stop bind
  • /usr/local/etc/named.conf
  • /usr/local/var/named/
  • rndc -k /usr/local/etc/rndc.key -p 54 status
  • named-checkconf -z /usr/local/etc/named.conf
  • host -t ns example.com
  • tail -f -n 40 /usr/local/var/log/named/named.log
Zone file specials:

example.net.  10800 IN SOA    ns1.example.net. xxx.example.net. (
                                                  2019090801
                                                  21600
                                                  900
                                                  1209600
                                                  3600)
                      10800 IN NS      ns1.example.net.
                      10800 IN NS      ns2.example.net.
                      10800 IN NS      ns3.example.net.
                      10800 IN A        111.111.111.111
                      10800 IN MX     10 smtp.example.net.
                      10800 IN TXT    "v=spf1 +mx -all"
                      10800 IN CAA    128 issue "letsencrypt.org"

Network Performance tools
See:
brew install iperf iperf3 nuttcp bwctl owamp
iperf
Server:
  • iperf -s -i 4 -w 5m
Client:
  • iperf -i 4 -t 1000 -N -w 5M -l 1M -c [servername]
iperf3
Server:
  • iperf3 -s -i 5
Client:
  • iperf3 -i 4 -t 1000 -N -w 5M -l 1M -c [servername]
nuttcp
On the server:
  • nuttcp -S
Client:
  • nuttcp [servername]
This runs a 10 second test, only on ipv4

SSH on a different port number
This procedure and port numbers come from:
Procedure:
  • sudo vi /etc/services
    • Change the port number in:
    • ssh 22/udp # SSH Remote Login Protocol
    • ssh 22/tcp # SSH Remote Login Protocol
  • sudo vi /etc/ssh/ssh_config
    • uncomment the following: Port 22
      and change that port number in the desired one.
  • Restart the ssh daemon.
  • sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
  • sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist

PHP
Note: DOES NOT WORK BECAUSE OF mod_mpm_event.so incompatible caching regimes!
  • brew install php
To enable PHP in Apache add the following to httpd.conf and restart Apache:

    LoadModule php7_module /usr/local/opt/php/lib/httpd/modules/libphp7.so

    <FilesMatch \.php$>
        SetHandler application/x-httpd-php
    </FilesMatch>

Finally, check DirectoryIndex includes index.php
  • bbedit DirectoryIndex index.php index.html
The php.ini and php-fpm.ini file can be found in:
    /usr/local/etc/php/7.3/

To have launchd start php now and restart at login:
  • brew services start php
Or, if you don't want/need a background service you can just run:
  • php-fpm

MacMini Setup (for SC/webcam/demo)
  • Users & Groups
    • SNE-Admin
      • sne-admin
    • SNE-demo
      • sne-demo
    • Login Options
      • auto login sne-demo
  • Power settings
    • never sleep computer
    • restart after power fail
    • prevent display sleep
    • awake with net access
    • start up 8h00 in the morning
  • Desktop & Screen Saver
    • no screen saver
  • Sharing
    • Screen Sharing
    • File Sharing
    • Remote login
  • Security
    • turn off screen lock
    • enable location services
    • no filevault
  • Date & Time
    • automatic time adjustment
  • Display Menu.app and WebCamMonitor App
  • MenuMeters
  • BBEdit
  • Deskpicture SNE logo
  • Team Viewer Setup
  • EvoCam Setup
    • Evocam5 download
      • serial ES56-MUDX-9LD6-BRAG
      • Note:
        • EvoCam 4 crashes now and then but does recording fine!
        • EvoCam 5 is more stable but gives unusable recordings!
        • note that sometimes the high res recording looks like taken at low res!
          Make sure to open first the small and then the big window.
    • Settings
      • Preferences
        • web server port nr 10456
        • Log Web Server access
        • auto-open docs from previous session at startup
        • make sure the low res is in the back.
        • Finder Cam1.settings on desktop put in dock and set Open at login
        Cam1.evocamsettings
        • 320 * 180
        • 384 * 216
        • 480 * 270
        • font size 12
        • framerate 15
        • quality normal normal
        • fontsize 9
      • Cam2.evocamsettings
        • 1280 * 720
        • framerate 15
        • quality normal normal
      • Other resolutions 16*9
      • 256    144    YouTube 144p
      • 426    240   
      • 640    360    nHD
      • 768    432   
      • 800    450   
      • 848    480   
      • 896    504   
      • 960    540    qHD
      • 1024    576   
      • 1152    648   
      • 1280    720    HD
      • 1366    768    WXGA
      • 1600    900    HD+
      • 1920    1080    Full HD
      • 2000    1125   
      • 2048    1152   
      • 2304    1296   
      • 2560    1440    QHD
      • 2880    1620   
      • 3200    1800    QHD+
      • 3520    1980   
      • 3840    2160    4K UHD
      • 4096    2304    Full 4K UHD
      • 4480    2520   
      • 5120    2880    5K UHD
      • 5760    3240   
      • 6400    3600   
      • 7040    3960   
      • 7680    4320    8K UHD
      • 15360    8640    16K
      • NOTE: the low resolution serve must start first, then the high resolution, otherwise video is low quality.

TimeMachine Problem Solving
The procedures come from:
http://www.garth.org/archives/2011,08,27,169,fix-time-machine-sparsebundle-nas-based-backup-errors.html

This procedure is to correct errors like: “Time Machine completed a verification of your backups. To improve reliability, Time Machine must create a new backup for you.”. It may or may not work.

The steps:
  • sudo chflags -R nouchg /Volumes/<PathTo+Name>.sparsebundle
  • sudo hdiutil attach -nomount -noverify -noautofsck /Volumes/<PathTo+Name>.sparsebundle
  • sudo tail -f /var/log/fsck_hfs.log
  • sudo fsck_hfs -drfy -c 8192M /dev/diskXs2
If you get a message in the fsck_hfs.log along the lines of " RebuildBTree – record x in node y is not r" then try:
  • fsck_hfs -p /dev/diskXs2
  • fsck_hfs -drfy -c 8192M  /dev/diskXs2

To scan for bad blocks:

  • /sbin/fsck_hfs -S /dev/diskXs2
When succeeded:
  • hdiutil detach /dev/diskXs2
When complete, you need to edit an plist file within the sparsebundle that records the state of the backup. On the top level of the sparsebundle find a file called "com.apple.TimeMachine.MachineID.plist". Edit it and remove these two nodes:

<key>RecoveryBackupDeclinedDate</key>

<date>{whatever-the-date}</date>

Finally you want to change:

<key>VerificationState</key>
<integer>2</integer>
to
<key>VerificationState</key>
<integer>0</integer>

Now you can eject the network share and have Time Machine give it another go. After the (long) verification step, backups should proceed once again.

How this page is made