Repeater linking: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
Brian Wilson (talk | contribs)
 
(61 intermediate revisions by the same user not shown)
Line 1: Line 1:
I am testing network configurations for TARRA, the Teton Amateur Radio Repeater Association in Wyoming.
I am testing configurations for TARRA, the '''Teton Amateur Radio Repeater Association''' in Wyoming.


Goal here is to route our 44 subnet to the repeaters. The repeaters can be on any
== IRLP and Pi Repeater stuff from W7BU ==
service provider so we need to accommodate that.
 
Mike has this stuff right now. 2022-11-18
* [https://irlp.net/ IRLP kit]
* Raspberry Pi 3B in a transparent case + [https://elecrow.com/wiki/index.php?title=HDMI_Interface_5_Inch_800x480_TFT_Display Elecrow 5" touch screen] plugs in directly to a Pi 3.
* A bunch of 5V wall warts with USB micro connectors
* Mini USB Keyboard
* 3x USB Mouse
* a pre-programmed 32GB SD card from Canakit
* 7" Composite LCD
* 2 or so random 12V wall warts with coax connectors
* 10BT patch cable
 
Brian has this (on loan)
* [https://wiki.tarra.link/index.php/Pi-Repeater-2X Pi Repeater 2X] from [https://ics-ctrl.com/pi-repeater/ ICS controllers] which uses [[SVXLink]]
* 1 12V/2.1A wall wart with coax connector
* Pi Zero W in transparent Vilros case, with several lids including one for a camera
* A weird bracket thing that might hold a LCD screen
 
The PI-REPEATER-2X is in the factory sheet metal box containing a PI-REPEATER-2X controller and a RPI 3B with the cabling done to bring out DB connectors.
It has a 12->5V DC regulator too. 2022-11 At some point the cheesy little voltage regulator failed and now I have an external 5V/5A Meanwell supply on it.
 
The puny failed supply was tiny. Not sure what to do to about that, since the replacement is not.
 
I also have a Meanwell PSD-30A-5 which needs JST connectors, pins are JST SVH 21T-P1.1 and the housings are VHR-3N and VHR-4N
 
== Two radios ==
 
I have two [[Kenwood TM-271A]] radios and I am looking at what I can do with them.
 
== NiceRF radios ==
 
SHARI -- Kits based on a VHF|UHF radio, set up for ASL, basically a hotspot. Based on the NiceRF SA818S.
 
The Pi 3 version ($65) connects via 2 USB connectors, so it's entirely driven off a serial port via USB.
There is also Pi 4 compatible version of this product.
 
The separate Pi Hat 4 ($80) version plugs into the GPIO connectors I think. It also requires soldering wires to the Pi 4. Ick.
 
Another SA818S - based thingie https://wb6amt.com/sa-818-carrier-board/ more generic than the SHARI but cheaper
 
Maker: https://kitsforhams.com/
 
Review, including a Youtube!! https://qrznow.com/shari-pi-hat-allstar-sa818-radio-module-for-raspberry-pi/
 
 
== AllStarLink ==


I have to keep in mind that the bigger picture is to control and link the repeaters,
We evaluated this and decided it's far too complex for this project. Still and all there is a page now [[All Star Link]]
so that might mean changing out the operating system. For example, the Pi image
distributed for Allstar is ArchLinux.


Wireguard would be one approach but
== Network routing and Wireguard ==
my current thought is to keep it as simple as possible by using only tunnels.


== Test setup #1 ==
Goal here is to route our 44 subnet to the repeaters. The repeaters can be on any
service provider so we need to accommodate that.


I am using a Pi4 and a Pi3 for testing right now, using the official image based on Debian.
I spent too much time researching ipip and gre tunnels and gave up and came back to Wireguard.
There might or might not be firewalls and NAT on some nodes, and certainly that is the case
here at home.


'''Violet''' is the pi3, connected over Wifi so I can ssh into it
Regarding IPIP and GRE though the best doc I have found is https://wiki.buyvm.net/doku.php/ipip_tunnel I got a tunnel running between two VPSs,  
tarra and w6gkd but I don't need a setup like that.


'''Tenrec''' is the pi4, connected by a 10BT patch cable to violet. Tenrec has a 7" screen and kbd.
So Wireguard it is.


Tools you will be be needing.
Instructions for setting up a Raspberry Pi as a client [[Wireguard client set up]]


apt install tcpdump
Install it,


=== IPIP tunnels ===
sudo apt-get install wireguard -y


cat /etc/modules-load.d
Instructions and download are available from
cat > ip_tunnel.conf
https://upcloud.com/community/tutorials/get-started-wireguard-vpn/
tunnel4
ipip
reboot


Now you have an unconfigured interface called tunl0. I add a new one, tun0
For the ERX router,
https://github.com/WireGuard/wireguard-vyatta-ubnt/wiki/EdgeOS-and-Unifi-Gateway


On violet,
=== Test setup ===
ip tunnel add tun0 mode ipip remote 192.168.1.2
ip addr add 44.127.9.2/24 dev tun0
ip link set tun0 up


On tenrec,
I am using a Pi3 and a VPS for testing right now, using the official image based on Debian.
ip tunnel add tun0 mode ipip remote 192.168.1.1
ip addr add 44.127.9.1/24 dev tun0
ip link set tun0 up


ping 44.127.9.1
'''Violet''' is the pi3, on my Spectrum broadband behind a Ubiquiti router.


== Test setup #2 ==
'''TARRA''' is the VPS, at VULTR.


Two virtual machines are used. This simulates a Pi deployed someplace on a Comcast link. (Or any other ISP, or even a wifi link, really.)
/etc/wireguard/wg0.conf is the config at each end


'''Tarra''' will be the router in deployment.
Bring up connection


'''W6gkd''' will be the "repeater"; in reality it's a virtual machine at a different service provider.
wg-quick up wg0


Test connection


I set up tunnels and tested them. Some notes on that, similar to the above. I created the module load file, it required one more module on tarra.
Shut down connection
cat /etc/modules-load.d
cat > ip_tunnel.conf
tunnel4
ip_tunnel
ipip
I did not reboot, I just used insmod to load them, trusting that the conf file will actually work on next reboot.


Then I did create a script (and run it) like this. It's called "ipip.sh" and it's in the root folder to remind me to sudo first. The 108.x IP is for w6gkd.
wg-quick down wg0


ip tunnel add tun0 mode ipip remote 108.161.129.155
Subnets https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=28&cip=44.127.9.0&ctype=ipv4&printit=0&x=66&y=16
ip addr add 44.127.9.2/24 dev tun0
ip link set tun0 up
route


With a similar set up on w6gkd I can ping from tarra to w6gkd with "ping 44.127.9.2" but
Show me the INPUT rules, verbosely
no echo comes back presumably because of routing issues. I can see packets pop out
iptables -L INPUT -v
of the tunnel with tcpdump at the far end.


In test or deployment, at this point I need the [https://wiki.ampr.org/wiki/Ampr-ripd AMPR router daemon] installed.
"ACCEPT" in this case says, "nothing interesting here", don't log.


== "Other" ==
On violet, you can log traffic to monitor it, or just use tcpdump
iptables -F INPUT
iptables -A INPUT -i wg0 -j LOG
iptables -F OUTPUT
iptables -A OUTPUT -i wg0 -j LOG
tail -f /var/log/messages
or
tcpdump -i wg0 -n


I also have tried GRE tunnels and Wireguard. Wireguard is actually deployed
on tarra but not used for the repeater links.


=== GRE tunnels ===
On tarra
tcpdump -i wg0 -n


Not working the way I expect,
# Make packets coming in from the Internet get written to the right subnet
iptables -t nat -A POSTROUTING -o wg0 -j DNAT -d 44.127.9.2


Basics
I think wireguard does this automatically
ip route add 44.127.9.0/28 via 44.127.9.1


* https://david-waiting.medium.com/a-beginners-guide-to-generic-routing-encapsulation-fb2b4fb63abb
=== Firewall settings ===
* https://www.xmodulo.com/create-gre-tunnel-linux.html


Types and basic commands
apt-get install tcpdump dnsutils iptables-persistent ipset fail2ban lynx git


On Raspbian I had to create a file to load the modules at boot, in this order.
I had fail2ban installed already on both machines, which means that iptables was also installed already and could be the whole problem. My iptables skills are rusty.


cd /etc/modules-load.d
"iptables -L" shows me that about 100 sites have been ssh banned. It also told me that FORWARD was DROP on w6gkd hmmm.
cat > gre_tunnel.conf
gre
ip_tunnel
ip_gre


I reboot at this point and make sure the modules are loading, with
iptables -A INPUT -p 4 -j ACCEPT
iptables -A INPUT -p udp --dport 520 -j ACCEPT
iptables -P FORWARD ACCEPT
# Drop various services we don't want running over the tunnel, mostly Microsoft stuff
iptables -A OUTPUT -o tun0 -p udp --dport 10001 -j DROP
iptables -A OUTPUT -o tun0 -p udp --dport 137:139 -j DROP
iptables -A OUTPUT -o tun0 -p udp --dport 5678 -j DROP
# Drops destination unreachable replies to various probe responses saving bandwidth
iptables -A OUTPUT -o tun0 -p icmp --icmp-type destination-unreachable -j DROP
# This prevents nested ipencap see https://ohiopacket.org/xrpi/docs/ipencap.htm
iptables -t raw -I PREROUTING -p 4 -i tun0 -j DROP
# This prevents a general loop
iptables -I FORWARD -i tun0 -o tun0 -j DROP
# Drops outbound unassigned IPs from looping though tunl0 via ipencap
# You must add accept rules under this line to make exceptions
# Drop traffic that does not have one of our 44 addresses on it.
iptables -I FORWARD ! -s 44.127.9.0/24 -o tunl0 -j DROP


  lsmod | grep gre
  # I don't think this will hurt anything but might no longer matter with current amprd 3.0
iptables -A OUTPUT -o ens3 -p icmp --icmp-type destination-unreachable -m state --state RELATED -j DROP


'''TUN interface''' - encapsulates ether header
=== Tarra server ===


The "gre0" interface exists so if I try to use the first command with gre0
Scripts are in /etc/wireguard/ to bring up connections including wg-all.sh
I get an 'exists' error,
I could follow the first example above and use "tun0" instead of "gre0"?


On violet,
Each remote node has its own set of keys in /etc/wireguard/KEYS and its own script, for example,
ip tunnel add tun0 mode gre remote 172.16.123.1 local 172.16.123.2 ttl 255
ip addr add 10.10.10.1/24 dev tun0
ip link set tun0 up


On tenrec, the other way round,
cd /etc/wireguard
  ip tunnel add tun0 mode gre remote 172.16.123.2 local 172.16.123.1 ttl 255
  ./wg-rendezvous.sh down
  ip addr add 10.10.10.2/24 dev tun0
  ./wg-rendezvous.sh up
ip link set tun0 up


To shutdown simply use, then press on and test TAP.
Restoring connections after rebooting is handled via systemd.
ip link set tun0 down


'''TAP interface''' - no ether header
Run "systemctl start wg-all.service" to bring everything up and
"systemctl stop wg-all.service" to stop everything. Check /var/log/daemon.log for messages.


On violet,
To implement this I created two files,  
ip link add tun1 type gretap remote 172.16.123.1 local 172.16.123.2 dev eth0
ip addr add 10.10.10.1/24 dev tun1
ip link set tun1 up
ip -d link show tun1


On tenrec, going the other direction,
  cd /lib/systemd/system
  ip link add tun1 type gretap remote 172.16.123.2 local 172.16.123.1 dev eth0
  cat wg-all.target
  ip addr add 10.10.10.2/24 dev tun1
  [Unit]
  ip link set tun1 up
  Description=WireGuard Tunnels for Tarra
  ip -d link show tun1


I need some sample commands here to confirm the links actually work.
and


  ping 10.10.10.1
  cat wg-all.service
  ping 172.16.123.1
[Unit]
Description=WireGuard via wg-all for TARRA
After=network-online.target nss-lookup.target
  Wants=network-online.target nss-lookup.target
PartOf=wg-all.target
   
   
  tcpdump -i tun0
  [Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/etc/wireguard/wg-all.sh up
ExecStop=/etc/wireguard/wg-all.sh down
[Install]
WantedBy=multi-user.target


== Wireguard ==
== Monitoring Wireguard ==
I tried building prometheus-wireguard-exporter for arm64 and gave up, then I found there is a pre-built version on the Docker Hub.


Wireguard is an encrypted tunnel that is easy to set up.
docker pull mindflavor/prometheus-wireguard-exporter:latest


Instructions and download are available from
This works to start it up, and to test it,
https://github.com/WireGuard/wireguard-vyatta-ubnt/wiki/EdgeOS-and-Unifi-Gateway


docker run -d --net=host --cap-add=NET_ADMIN --name wgexporter mindflavor/prometheus-wireguard-exporter -a true
curl -s http://localhost:9586/metrics


[[Category: Radio]]
[[Category: Radio]]
[[Category: Network]]
[[Category: Network]]
[[Category: System Administration]]
[[Category: System Administration]]

Latest revision as of 03:57, 11 July 2024

I am testing configurations for TARRA, the Teton Amateur Radio Repeater Association in Wyoming.

IRLP and Pi Repeater stuff from W7BU

Mike has this stuff right now. 2022-11-18

  • IRLP kit
  • Raspberry Pi 3B in a transparent case + Elecrow 5" touch screen plugs in directly to a Pi 3.
  • A bunch of 5V wall warts with USB micro connectors
  • Mini USB Keyboard
  • 3x USB Mouse
  • a pre-programmed 32GB SD card from Canakit
  • 7" Composite LCD
  • 2 or so random 12V wall warts with coax connectors
  • 10BT patch cable

Brian has this (on loan)

  • Pi Repeater 2X from ICS controllers which uses SVXLink
  • 1 12V/2.1A wall wart with coax connector
  • Pi Zero W in transparent Vilros case, with several lids including one for a camera
  • A weird bracket thing that might hold a LCD screen

The PI-REPEATER-2X is in the factory sheet metal box containing a PI-REPEATER-2X controller and a RPI 3B with the cabling done to bring out DB connectors. It has a 12->5V DC regulator too. 2022-11 At some point the cheesy little voltage regulator failed and now I have an external 5V/5A Meanwell supply on it.

The puny failed supply was tiny. Not sure what to do to about that, since the replacement is not.

I also have a Meanwell PSD-30A-5 which needs JST connectors, pins are JST SVH 21T-P1.1 and the housings are VHR-3N and VHR-4N

Two radios

I have two Kenwood TM-271A radios and I am looking at what I can do with them.

NiceRF radios

SHARI -- Kits based on a VHF|UHF radio, set up for ASL, basically a hotspot. Based on the NiceRF SA818S.

The Pi 3 version ($65) connects via 2 USB connectors, so it's entirely driven off a serial port via USB. There is also Pi 4 compatible version of this product.

The separate Pi Hat 4 ($80) version plugs into the GPIO connectors I think. It also requires soldering wires to the Pi 4. Ick.

Another SA818S - based thingie https://wb6amt.com/sa-818-carrier-board/ more generic than the SHARI but cheaper

Maker: https://kitsforhams.com/

Review, including a Youtube!! https://qrznow.com/shari-pi-hat-allstar-sa818-radio-module-for-raspberry-pi/


AllStarLink

We evaluated this and decided it's far too complex for this project. Still and all there is a page now All Star Link

Network routing and Wireguard

Goal here is to route our 44 subnet to the repeaters. The repeaters can be on any service provider so we need to accommodate that.

I spent too much time researching ipip and gre tunnels and gave up and came back to Wireguard. There might or might not be firewalls and NAT on some nodes, and certainly that is the case here at home.

Regarding IPIP and GRE though the best doc I have found is https://wiki.buyvm.net/doku.php/ipip_tunnel I got a tunnel running between two VPSs, tarra and w6gkd but I don't need a setup like that.

So Wireguard it is.

Instructions for setting up a Raspberry Pi as a client Wireguard client set up

Install it,

sudo apt-get install wireguard -y

Instructions and download are available from https://upcloud.com/community/tutorials/get-started-wireguard-vpn/

For the ERX router, https://github.com/WireGuard/wireguard-vyatta-ubnt/wiki/EdgeOS-and-Unifi-Gateway

Test setup

I am using a Pi3 and a VPS for testing right now, using the official image based on Debian.

Violet is the pi3, on my Spectrum broadband behind a Ubiquiti router.

TARRA is the VPS, at VULTR.

/etc/wireguard/wg0.conf is the config at each end

Bring up connection

wg-quick up wg0

Test connection

Shut down connection

wg-quick down wg0

Subnets https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=28&cip=44.127.9.0&ctype=ipv4&printit=0&x=66&y=16

Show me the INPUT rules, verbosely

iptables -L INPUT -v

"ACCEPT" in this case says, "nothing interesting here", don't log.

On violet, you can log traffic to monitor it, or just use tcpdump

iptables -F INPUT
iptables -A INPUT -i wg0 -j LOG
iptables -F OUTPUT
iptables -A OUTPUT -i wg0 -j LOG
tail -f /var/log/messages

or

tcpdump -i wg0 -n


On tarra

tcpdump -i wg0 -n
# Make packets coming in from the Internet get written to the right subnet
iptables -t nat -A POSTROUTING -o wg0 -j DNAT -d 44.127.9.2

I think wireguard does this automatically

ip route add 44.127.9.0/28 via 44.127.9.1

Firewall settings

apt-get install tcpdump dnsutils iptables-persistent ipset fail2ban lynx git

I had fail2ban installed already on both machines, which means that iptables was also installed already and could be the whole problem. My iptables skills are rusty.

"iptables -L" shows me that about 100 sites have been ssh banned. It also told me that FORWARD was DROP on w6gkd hmmm.

iptables -A INPUT -p 4 -j ACCEPT
iptables -A INPUT -p udp --dport 520 -j ACCEPT
iptables -P FORWARD ACCEPT

# Drop various services we don't want running over the tunnel, mostly Microsoft stuff
iptables -A OUTPUT -o tun0 -p udp --dport 10001 -j DROP
iptables -A OUTPUT -o tun0 -p udp --dport 137:139 -j DROP
iptables -A OUTPUT -o tun0 -p udp --dport 5678 -j DROP 
# Drops destination unreachable replies to various probe responses saving bandwidth
iptables -A OUTPUT -o tun0 -p icmp --icmp-type destination-unreachable -j DROP

# This prevents nested ipencap see https://ohiopacket.org/xrpi/docs/ipencap.htm
iptables -t raw -I PREROUTING -p 4 -i tun0 -j DROP
# This prevents a general loop
iptables -I FORWARD -i tun0 -o tun0 -j DROP
# Drops outbound unassigned IPs from looping though tunl0 via ipencap
# You must add accept rules under this line to make exceptions
# Drop traffic that does not have one of our 44 addresses on it.
iptables -I FORWARD ! -s 44.127.9.0/24 -o tunl0 -j DROP
# I don't think this will hurt anything but might no longer matter with current amprd 3.0
iptables -A OUTPUT -o ens3 -p icmp --icmp-type destination-unreachable -m state --state RELATED -j DROP

Tarra server

Scripts are in /etc/wireguard/ to bring up connections including wg-all.sh

Each remote node has its own set of keys in /etc/wireguard/KEYS and its own script, for example,

cd /etc/wireguard
./wg-rendezvous.sh down
./wg-rendezvous.sh up

Restoring connections after rebooting is handled via systemd.

Run "systemctl start wg-all.service" to bring everything up and "systemctl stop wg-all.service" to stop everything. Check /var/log/daemon.log for messages.

To implement this I created two files,

cd /lib/systemd/system
cat wg-all.target
[Unit]
Description=WireGuard Tunnels for Tarra

and

cat wg-all.service 
[Unit]
Description=WireGuard via wg-all for TARRA
After=network-online.target nss-lookup.target
Wants=network-online.target nss-lookup.target
PartOf=wg-all.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/etc/wireguard/wg-all.sh up
ExecStop=/etc/wireguard/wg-all.sh down

[Install]
WantedBy=multi-user.target

Monitoring Wireguard

I tried building prometheus-wireguard-exporter for arm64 and gave up, then I found there is a pre-built version on the Docker Hub.

docker pull mindflavor/prometheus-wireguard-exporter:latest

This works to start it up, and to test it,

docker run -d --net=host --cap-add=NET_ADMIN --name wgexporter mindflavor/prometheus-wireguard-exporter -a true
curl -s http://localhost:9586/metrics