Setup Web Server (apache) and DNS (bind) on a MacMini using Homebrew

Contents

  1. Online references
  2. Common pitfalls, problems, usefull commands
  3. XCODE - command line tools
  4. BREW
  5. APACHE
  6. HTTPD config edits including HTTP2 and vhosts
  7. HTML default header
  8. Let's Encrypt - certbot
  9. Named and bind
  10. Network Performance tools
  11. SSH on a different port number
  12. PHP
  13. MacMini Setup (for SC/webcam/demo)
  14. TimeMachine Problem Solving
  15. How this page is made
S=Setup, O=Operational, T=Test

Online references

In this table are the commands for apache, bind and certbot via brew as in https://getgrav.org/blog/macos-mojave-apache-multiple-php-versions.
The previous Apple server software needs to be completely uninstalled and out of the window!

Common pitfalls, problems, usefull commands

  • After a brew upgrade some services may not work. One difficult to diagnose problem may be the firewall settings in MacOSX. It blocks by default incoming connections on unsigned bin's and executables need to be added in the list in the System Preferences Firewall settings. Also note that just putting there the /usr/local/bin/executable will not work since brew puts aliasses (links) there. Follow the link to the real bin. Same for /usr/local/sbin .
  • After a "brew upgrade" the output may display an error that an old directory could not be removed and a sudo command is displayed. In reality the "brew upgrade" did not complete, so execute that sudo command and repeat "brew upgrade" and displayed error - sudo commands until nothing is being done anymore by that command.
  • Somehow launchd has a different or incomplete PATH environment. Caused certbot renew to initially fail.
  • if you need to log out another user:
    • ps awwwwux | grep loginwindow
    • kill the pid of that user
  • Support files can be found HERE.

XCODE - command line tools
S xcode-select --install Apple's command line tools need to be installed, XCode is not needed

BREW
S ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
T brew --version https://docs.brew.sh/FAQ
O brew update
O brew upgrade to upgrade all installed programs to newest version.
O brew doctor
O sudo chown -R "$USER":admin /usr/local
sudo chown -R "$USER":admin /Library/Caches/Homebrew		 to correct prmissions
S brew install openldap libiconv

APACHE
S sudo apachectl stop
S sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
S brew install httpd
O sudo brew services start httpd
T ps -aef | grep httpd
O sudo brew services restart httpd
T tail -f /usr/local/var/log/httpd/error_log
O sudo apachectl stop
O sudo apachectl -k restart
T sudo apachectl -S to get setup paths
T sudo apachectl configtest test conf files, must be sudo to test certificates
S bbedit /usr/local/etc/httpd/httpd.conf

HTTPD config edits including HTTP2 and vhosts
S HTTPD config basic edits:
  1. Listen 8080 => Listen 80
  2. enable ==> LoadModule deflate_module lib/httpd/modules/mod_deflate.so
  3. enable ==> LoadModule rewrite_module lib/httpd/modules/mod_rewrite.so
  4. ServerAdmin admin@domain.net
  5. #ServerName www.example.com:8080 ==> ServerName localhost:80
  6. enable ==> Include /usr/local/etc/httpd/extra/httpd-autoindex.conf
S HTTPD config edits to enable http2
  1. disable ===> #LoadModule mpm_prefork_module lib/httpd/modules/mod_mpm_prefork.so
  2. enable ===> LoadModule mpm_event_module lib/httpd/modules/mod_mpm_event.so
  3. enable ===> LoadModule http2_module lib/httpd/modules/mod_http2.so
  4. add ===> Protocols h2 h2c http/1.1
S HTTPD config edits enable vhosts:
  1. enable ==> LoadModule vhost_alias_module lib/httpd/modules/mod_vhost_alias.so
  2. enable ==> Include /usr/local/etc/httpd/extra/httpd-vhosts.conf
S and for vhosts including a catch directory :

edit: /usr/local/etc/httpd/extra/httpd-vhosts.conf

<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
</VirtualHost>
<Directory "/usr/local/var/www">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

<VirtualHost *:80>
    DocumentRoot "/Users/example/Sites"
    ServerName example.com
    ServerAlias www.example.com
</VirtualHost>
<Directory "/Users/example/Sites">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

 The catch directory is the default webroot where all requests go to if there is not a specific webroot for that (sub)domain defined. Therefore, it must be the first in the list. Here an example for example.com

HTML default header

Web pages header:

<!DOCTYPE html>
<html>
  <head>
    <title>title</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=0.25, maximum-scale=4.0, user-scalable=yes">
    <meta name="description" content="title">
    <meta name="author" content="Cees de Laat">
  </head>
  <body style="font-family: Helvetica,Arial,sans-serif;" text="#ffffff" bgcolor="#000000" link="#ffcc66" alink="#ff9900" vlink="#ffff66">

Let's Encrypt - certbot

S sudo install -d -o $(whoami) -g admin /usr/local/Frameworks
S brew install certbot
S HTTPD edits: Uncomment:
  • LoadModule ssl_module modules/mod_ssl.so
  • LoadModule socache_shmcb_module modules/mod_socache_shmcb.so

add at the end:

<IfModule mod_ssl.c>
   Listen 443
</IfModule>
Include /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf
S in httpd-vhosts.conf:

<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
</VirtualHost>
<Directory "/usr/local/var/www">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

<VirtualHost *:80>
    DocumentRoot "/usr/local/var/www"
    ServerName example.com
    ServerAlias www.example.com
RewriteEngine on
RewriteCond %{SERVER_NAME} =example.com [OR]
RewriteCond %{SERVER_NAME} =www.example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<Directory "/Users/example/Sites">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>
and create/edit httpd-vhosts-le-ssl.conf:

<VirtualHost *:443>
    DocumentRoot "/usr/local/var/www"
    ServerName catch.example.com
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
</VirtualHost>

<VirtualHost *:443>
    DocumentRoot "/Users/example/Sites"
    ServerName example.com
    ServerAlias www.example.com
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
</VirtualHost>
 The idea is that all requests that get through on port 80 or with incorrect url's/domain names end up in:
  • "/usr/local/var/www"
and all that comes in with https and correct domain names go to the correct webroots of those domains.

For that purpose different rewrite rules take care of redirection. Those are in
httpd-vhosts.conf
I use the following domain construction for example.com:
  • catch example.com
    • catches all (sub)domains for which no other webroot is defined is
  • example.com
    • this goes to the normal webroot of the domain
  • www.example.com
    • also to the typical webroot of example.com
  • sub1.example.com
    • a subdomain of example.com with its own webroot
S sudo certbot --apache
or
sudo certbot --apache --staging
and after successful testing:
sudo certbot --apache --force-renewal		 conf file: /etc/letsencrypt/renewal/
O sudo certbot renew --dry-run
O sudo certbot renew for production
O sudo certbot certificates
S For renewal we have to make a shell script and a launchd plist.

create file /usr/local/etc/certbot-renew.sh :

touch /usr/local/etc/certbot-renew.sh

#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
sudo certbot renew

chmod +x /usr/local/etc/certbot-renew.sh

/Library/LaunchDaemons/com.letsencrypt.renew.plist

sudo vi /Library/LaunchDaemons/com.letsencrypt.renew.plist
put in that file the xml below here:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
    <string>com.letsencrypt.renew</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/etc/certbot-renew.sh</string>
    </array>
   <key>StandardErrorPath</key>
    <string>/tmp/com.letsencrypt.renew.stderr</string>
    <key>StandardOutPath</key>
    <string>/tmp/com.letsencrypt.renew.stdout</string>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>4</integer>
        <key>Minute</key>
        <integer>56</integer>
    </dict>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>
 https://gist.github.com/jgillman/8b29c5d75f9fbcf30d55f1a0de535c5d

Only the plist in the above solution does not work because somehow the PATH variable of the running deamon is not correct. Therefore we make our own script that sets the path and invokes certbot renew, and then we use a LaunchDeamon to periodically invoke the script.
S sudo chmod 644 /Library/LaunchDaemons/com.letsencrypt.renew.plist
sudo chown root:admin /Library/LaunchDaemons/com.letsencrypt.renew.plist
sudo launchctl load /Library/LaunchDaemons/com.letsencrypt.renew.plist
sudo launchctl list | grep -i letsencrypt
T tail -f -n 40 /tmp/com.letsencrypt.renew.stderr
tail -f -n 40 /tmp/com.letsencrypt.renew.stdout
sudo tail -f -n 400 /var/log/letsencrypt/letsencrypt.log

Named and bind

S brew install bind
O sudo brew services start bind
O sudo brew services restart bind
O sudo brew services stop bind
S /usr/local/etc/named.conf for the named.conf file
S /usr/local/var/named/ for the zone files
T rndc -k /usr/local/etc/rndc.key -p 54 status
T named-checkconf -z /usr/local/etc/named.conf
T host -t ns example.com
T
 tail -f -n 40 /usr/local/var/log/named/named.log

Zone file specials:

example.net.  10800 IN SOA    ns1.example.net. xxx.example.net. (
                                                  2019090801
                                                  21600
                                                  900
                                                  1209600
                                                  3600)
                      10800 IN NS      ns1.example.net.
                      10800 IN NS      ns2.example.net.
                      10800 IN NS      ns3.example.net.
                      10800 IN A        111.111.111.111
                      10800 IN MX     10 smtp.example.net.
                      10800 IN TXT    "v=spf1 +mx -all"
                      10800 IN CAA    128 issue "letsencrypt.org"

Network Performance tools
S
 brew install iperf iperf3 nuttcp bwctl owamp https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/

iperf
Server:
iperf -s -i 4 -w 5m

Client:
iperf -i 4 -t 1000 -N -w 5M -l 1M -c [servername]

iperf3
Server:
iperf3 -s -i 5

Client:
iperf3 -i 4 -t 1000 -N -w 5M -l 1M -c [servername]

nuttcp
Server:
nuttcp -S

Client:
nuttcp [servername]		 runs a 10 second test, only on ipv4



SSH on a different port number

sudo vi /etc/services

Change the port number in

ssh 22/udp # SSH Remote Login Protocol
ssh 22/tcp # SSH Remote Login Protocol

sudo vi /etc/ssh/ssh_config

uncomment the following: Port 22
and change that port number in the required one.

Restart the ssh daemon.
sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
 https://superuser.com/questions/1398824/cant-change-port-listen-on-macos-x-mojave-for-built-in-ssh

see:  http://www.iana.org/assignments/port-numbers

PHP
S brew install php DOES NOT WORK BECAUSE OF mod_mpm_event.so
incompatible caching regimes!
S To enable PHP in Apache add the following to httpd.conf and restart Apache:
    LoadModule php7_module /usr/local/opt/php/lib/httpd/modules/libphp7.so

    <FilesMatch \.php$>
        SetHandler application/x-httpd-php
    </FilesMatch>

Finally, check DirectoryIndex includes index.php
    DirectoryIndex index.php index.html

The php.ini and php-fpm.ini file can be found in:
    /usr/local/etc/php/7.3/

To have launchd start php now and restart at login:
  brew services start php
Or, if you don't want/need a background service you can just run:
  php-fpm

MacMini Setup (for SC/webcam/demo)

  • Users & Groups
    • SNE-Admin
      • sne-admin
    • SNE-demo
      • sne-demo
    • Login Options
      • auto login sne-demo
  • Power settings
    • never sleep computer
    • restart after power fail
    • prevent display sleep
    • awake with net access
    • start up 8h00 in the morning
  • Desktop & Screen Saver
    • no screen saver
  • Sharing
    • Screen Sharing
    • File Sharing
    • Remote login
  • Security
    • turn off screen lock
    • enable location services
    • no filevault
  • Date & Time
    • automatic time adjustment
  • Display Menu.app and WebCamMonitor App
  • MenuMeters
  • BBEdit
  • Deskpicture SNE logo
  • Team Viewer Setup
  • EvoCam Setup
    • Evocam5 download
      • serial ES56-MUDX-9LD6-BRAG
      • Note:
        • EvoCam 4 crashes now and then but does recording fine!
        • EvoCam 5 is more stable but gives unusable recordings!
        • note that sometimes the high res recording looks like taken at low res!
          Make sure to open first the small and then the big window.
    • Settings
      • Preferences
        • web server port nr 10456
        • Log Web Server access
        • auto-open docs from previous session at startup
        • make sure the low res is in the back.
        • Finder Cam1.settings on desktop put in dock and set Open at login
        Cam1.evocamsettings
        • 320 * 180
        • 384 * 216
        • 480 * 270
        • font size 12
        • framerate 15
        • quality normal normal
        • fontsize 9
      • Cam2.evocamsettings
        • 1280 * 720
        • framerate 15
        • quality normal normal
      • Other resolutions 16*9
      • 256    144    YouTube 144p
      • 426    240   
      • 640    360    nHD
      • 768    432   
      • 800    450   
      • 848    480   
      • 896    504   
      • 960    540    qHD
      • 1024    576   
      • 1152    648   
      • 1280    720    HD
      • 1366    768    WXGA
      • 1600    900    HD+
      • 1920    1080    Full HD
      • 2000    1125   
      • 2048    1152   
      • 2304    1296   
      • 2560    1440    QHD
      • 2880    1620   
      • 3200    1800    QHD+
      • 3520    1980   
      • 3840    2160    4K UHD
      • 4096    2304    Full 4K UHD
      • 4480    2520   
      • 5120    2880    5K UHD
      • 5760    3240   
      • 6400    3600   
      • 7040    3960   
      • 7680    4320    8K UHD
      • 15360    8640    16K
      • NOTE: the low resolution serve must start first, then the high resolution, otherwise video is low quality.

TimeMachine Problem Solving


    chflags -R nouchg /Volumes/<PathTo+Name>.sparsebundle
    hdiutil attach -nomount -noverify -noautofsck /Volumes/<PathTo+Name>.sparsebundle
    tail -f /var/log/fsck_hfs.log
    fsck_hfs -drfy /dev/diskXs2

If you get a message in the fsck_hfs.log along the lines of
 RebuildBTree – record x in node y is not r
then try
    fsck_hfs -p /dev/diskXs2
    fsck_hfs -drfy  /dev/diskXs2

When succeeded:
    hdiutil detach /dev/diskXs2

When complete, you need to edit an plist file within the sparsebundle that records the state of the backup. On the top level of the sparsebundle find a file called com.apple.TimeMachine.MachineID.plist. Edit it and remove these two nodes

<key>RecoveryBackupDeclinedDate</key>

<date>{whatever-the-date}</date>

Finally you want to change
<key>VerificationState</key>
<integer>2</integer>
to
<key>VerificationState</key>
<integer>0</integer>

Now you can eject the network share and have Time Machine give it another go. After the (long) verification step, backups should proceed once again.
http://www.garth.org/archives/2011,08,27,169,fix-time-machine-sparsebundle-nas-based-backup-errors.html

How this page is made