Networking

How to find out architecture of your Mikrotik RouterOS router

Find out the CPU architecture using the webinterface (WebFig)

In the WebFig Web UI you can go to System -> Resources where you can see the architecture listed as Architecture name:

Find out the CPU architecture using the terminal

On the terminal, run

/system resource print

and look for the architecture-name line. In the following example, the architecutre is arm:

[admin@MyRouter] > /system resource print
                   uptime: 10m24s
                  version: 7.3.1 (stable)
               build-time: Jun/09/2022 08:58:15
         factory-software: 6.44.6
              free-memory: 446.0MiB
             total-memory: 512.0MiB
                      cpu: ARM
                cpu-count: 2
                 cpu-load: 0%
           free-hdd-space: 1148.0KiB
          total-hdd-space: 16.0MiB
  write-sect-since-reboot: 595
         write-sect-total: 139871
               bad-blocks: 0%
        architecture-name: arm
               board-name: CRS326-24G-2S+
                 platform: MikroTik

 

Posted by Uli Köhler in MikroTik, Networking

Which MikroTik switch can you use with 100M SFP modules?

Generally, 100M SFP modules can not be used with SFP+ ports. They sometimes can be used with SFP ports, however there is no guarantee it will work properly until you’ve actually tested the compatibility of the hardware!

Besides using a 100M SFP Module with a 100M-compatible SFP port, there is also the possibility of using a SFP Module with integrated converter. FS offers such a device for ~55€ but at the moment I do not know if it is compatible with an Mikrostil device.

Compatible devices

The MikroTik help page lists the CRS106-5S-1C as being compatible with both 100M and 1G SFP modules:

This unit is compatible with 100Mbit and 1.25G SFP modules

It has 5 SFP ports and 1 Combo SFP or GigE port.

Furthermore, the MikroTik wiki has a list of devices compatible with 100M fiberoptic transceivers – at the time of writing this post:

  • CCR1009-7G-1C
  • CCR1009-7G-1C-1S+
  • CRS106-1C-5S
  • CRS328-4C-20S-4S+
  • LHG XL 52 ac
  • RBD22/D23/mANTBox 52 15s/NetMetal ac²

Besides manually searching the MikroTik site for other compatible devices, I also used Google to search for similar sentences on the MikroTik site. I could not find any other MikroTik device for which any statement about 100Mbit SFP compatibility is being explicitly made.

Incompatible devices

For the following devices I have checked the respective MikroTik help page and it does not list compatibility with 100M SFP modules. This does not automatically mean they aren’t compatible but it’s much less likely. Possibly the help page will be updated in the future to indicate compatibility. I have not physically tested any of those devices with 100M transceivers.

  • CRS310-1G-5S-4S+IN
  • CRS112-8P-4S-IN
  • hEX S
  • CRS109-8G-1S-2HnD-IN
  • CRS212-1G-10S-1S+IN

Often, the help pages with read something like Compatible with 1.25G SFP modules. This means that standard 100Mbit SFP modules are incompatible.

Posted by Uli Köhler in Electronics, MikroTik, Networking

Which Ethernet PHY to use for 100Base-FX (SFP) operation?

For new designs I primarily recommend the Texas Instruments DP83822. It comes in a 5x5mm QFN package and provides RMII, MII and even RGMII interfaces to the Microcontroller or other Ethernet MAC.

Be sure to select the DP83822xF – the F means Fiber!

Since at the time of writing this article, the DP83822 has supply shortages, the following alternatives are available for 100Base-FX operation:

  • DP83869HM is a Gigabit Ethernet transceiver that supports 100Base-FX but does not support MII or RMII (only the Gigabit interfaces RGMII & SGMII)
  • DP83620 is a plain 10/100 PHY with RMII & fiber support
  • BCM5221 (MII & RMII) & BCM5241 (MII) are plain old 10/100 PHYs but in typical broadcom fashion, they don’t even give you the datasheet to download on their website. But you can find it via Google
  • BCM5248 is a 8-port PHY with fiber support
  • ST802RT1 (LQFP48)
  • Marvell’s 88E3015 & 88E3018 feature fiber support and have good documentation on the fiber interface. They are available in QFN packages but only support MII & RGMII – RMII is not supported !
  • KS8721BL, KS8721SL & KSZ8721CL (LQFP/SSOP)
  • KSZ8001L (LQFP/SSOP)
  • KSZ8041FTL (LQFP/SSOP – you must buy the FTL variant!)
  • (LQFP/SSOP)
  • The VSC8211 is a Gigabit Ethernet PHY but supports 100Base-FX & RMII interface. It is available in a 117-pin BGA package and hence rather difficult to use.
  • The LAN9355 3-port Ethernet switch features two 100Base-FX fiber interfaces and a RMII interface. It is more complex to use than a standalone PHY but can forward traffic without software interaction.
  • The KSZ8893FQL 3-port Ethernet switch features one 100Base-FX fiber interface and a RMII interface. It is rather expensive
  • The Cortina LXT971A is a simple 100Base-FX PHY from a rather unknown manufacturer. It only has MII, not RMII ! Cortina appears to have been bought by Intel.
  • LU3X34FTR is a 4-port 10/100 PHY with fiber support

Compared to the DP83822, within the context of 100Base-FX operation, there are few technical differences in whether you use the DP83822. In my experience, Ethernet PHYs are mostly difference with regards to their electrical immunity (ESD and so on) which is not really relevant in the fiber context unless someone directly touches the PCB, and the ability to compensate for a degrated Ethernet signal (which is not really relevant for fiber contexts). The only real difference between the DP83822 and many other parts is that the DP83822 comes in a small VQFN package, which the Micrel/Microchip KSZ devices come in much larger SSOP or LQFP packages. My recommendation is to select based on availability first, on size second.

Posted by Uli Köhler in Electronics, Networking

How to disable XCP-NG Windows Update PCIe device on the command line

This post shows you how to disable the XCP-NG windows update device on the command line. This prevents automatic installation of the Citrix drivers, enabling manual install of a custom version.

Note that you can easily disable the Windows update PCIe device in XenOrchestra using a single click, but not in XCP-NG center!

Prerequisite: Shut down the VM in question – usually you need to disable the device before installing Windows!

First, get the UUID of the VM usinjg

xe vm-list

which will output, for each virtual machine, something like:

uuid ( RO)           : 98002b8d-070f-9638-071c-be7e6c82f6a3
     name-label ( RW): CoreOS
    power-state ( RO): running

From that, copy the UUID such as 98002b8d-070f-9638-071c-be7e6c82f6a3.

Now run:

xe vm-param-set uuid=YOURUUID has-vendor-device=false

for example,

xe vm-param-set uuid=98002b8d-070f-9638-071c-be7e6c82f6a3 has-vendor-device=false

Now you can startup your VM with the driver installation PCIe device being disabled.

Posted by Uli Köhler in Networking, Virtualization

How to get router identity (name) in MikroTik RouterOS scripting

Use

[/system identity get name]

For example, you can use it like this:

/tool e-mail send [email protected] subject="My identity is $[/system identity get name]"

 

Posted by Uli Köhler in MikroTik, Networking

How to insert output of command into string in MikroTik RouterOS (scripting)

If you have a MikroTik RouterOS command such as

/tool e-mail send [email protected] subject="MikroTik test E-Mail"

you can insert the output of a command such as

/system identity get name

into it by using the $[...] syntax:

/tool e-mail send [email protected] subject="My identity is $[/system identity get name]"

 

Posted by Uli Köhler in MikroTik, Networking

How to setup Cloudflare DNS-over-HTTPS (DoH) cache on MikroTik RouterOS router

Compared to standard UDP DNS, DNS-over-HTTPS (DoH) provides the huge advantage that – due to it being encrypted, someone able to sniff the traffic will not be able to determine what domain names are being used.

However, consider the disadvantage that the latency of resolving a domain name is significantly larger with DoH – however, setting up the MikroTik router as DNS cache will significantly reduce the overall DNS latency, at least for cached domain names.

The following list of RouterOS commands will setup the internal DNS server as a DNS cache running on DNS-over-HTTPS.

First, download CA certificates onto the router in order to be able to verify CloudFlare’s HTTPS certificates:

/tool fetch url=https://curl.se/ca/cacert.pem

Wait for it to finish downloading, e.g.

[admin@MikroTik] > /tool fetch url=https://curl.se/ca/cacert.pem
      status: finished
  downloaded: 210KiBz pause]
       total: 210KiB
    duration: 1s

Now import the file and setup the DNS server:

/certificate import file-name=cacert.pem passphrase=""
/ip dns set allow-remote-requests=yes cache-size=8192KiB max-concurrent-queries=1000 max-concurrent-tcp-sessions=2000 servers=1.1.1.1 use-doh-server=https://cloudflare-dns.com/dns-query verify-doh-cert=yes

 

Posted by Uli Köhler in MikroTik, Networking

MikroTik webinterface reverse proxy using Traefik

The following Traefik .toml file which reverse proxies a MikroTik router’s WebFig webinterface is based on our Traefik setup from Simple Traefik docker-compose setup with Lets Encrypt Cloudflare DNS-01 & TLS-ALPN-01 & HTTP-01 challenges. It assumes that the MikroTik router is reachable at 10.1.2.3 via HTTP.

No Authentication beyond the MikroTik router’s WebFig internal authentication is performed. However – at least when using our Traefik config from our previous post it enforces HTTPS i.e. encrypted access.

Save the following file under config/mikrotik01.toml. Traefik will automatically reload, no restart will be required.

[http.routers.mikrotik01]
rule = "Host(`mikrotik01.mydomain.com`)"
service = "mikrotik01"

[http.routers.mikrotik01.tls]
certresolver = "cloudflare"

[[http.routers.mikrotik01.tls.domains]]
main = "mydomain.com"
sans = ["*.mydomain.com"]

[http.services]
[http.services.mikrotik01.loadBalancer]
[[http.services.mikrotik01.loadBalancer.servers]]
url = "http://10.1.2.3.4/"

 

Posted by Uli Köhler in MikroTik, Networking, Traefik

XenOrchestra docker-compose setup with Traefik labels

Based on Simple XenOrchestra setup using docker-compose, this extension of our config from that post features Traefik container labels. For the Traefik configuration, see for example our previous post Simple Traefik docker-compose setup with Lets Encrypt Cloudflare DNS-01 & TLS-ALPN-01 & HTTP-01 challenges

This setup uses a Wildcard certificate but you can also use a non-wildcard cert (e.g. if you don’t have access to the DNS for the DNS01 challenge) by just deleting both traefik.http.routers.xenorchestra.tls.domains... lines and selecting a suitable resolver.

version: '3'
services:
    xen-orchestra:
        restart: unless-stopped
        image: ronivay/xen-orchestra:latest
        container_name: xen-orchestra
        network_mode: host
        stop_grace_period: 1m
        environment:
            - HTTP_PORT=1780
        cap_add:
          - SYS_ADMIN
        security_opt:
          - apparmor:unconfined
        volumes:
          - ./xo-data:/var/lib/xo-server
          - ./redis-data:/var/lib/redis
        labels:
          - "traefik.enable=true"
          - "traefik.http.routers.xenorchestra.rule=Host(`xenorchestra.mydomain.com`)"
          - "traefik.http.routers.xenorchestra.entrypoints=websecure"
          - "traefik.http.routers.xenorchestra.tls.certresolver=cloudflare"
          - "traefik.http.routers.xenorchestra.tls.domains[0].main=mydomain.com"
          - "traefik.http.routers.xenorchestra.tls.domains[0].sans=*.mydomain.com"
          - "traefik.http.services.xenorchestra.loadbalancer.server.port=1780"

You can now login with the default credentials: [email protected] and password admin

Posted by Uli Köhler in Networking, Virtualization

nginx FritzBox webinterface reverse proxy

The following nginx config allows remote access to a local FritzBox over VPN etc. You explicitly need to set the Host header to fritz.box in the proxied request – else, the FritzBox will reject the request as part of its rebind protection.

server {
        listen 80 default_server;

        access_log off;
        error_log  off;
        location / {
            proxy_pass http://192.168.241.1;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host 'fritz.box';
        }
}

On most linux distributions such as Debian or Ubuntu, install nginx using sudo apt -y install nginx or similar and place our config file as /etc/nginx/sites-enabled/default.

Posted by Uli Köhler in Networking, nginx

How to permanently enable IPv4 forwarding in Alpine Linux

This simple command will permanently enable IPv4 forwarding on Alpine Linux. Run this as root:

echo net.ipv4.ip_forward=1 | tee -a /etc/sysctl.conf && sysctl -p

 

Posted by Uli Köhler in Alpine Linux, Networking

How to install tailscale on XCP-NG host

By installing tailscale on XCP-NG hosts, you can provide easier access to your virtualization host using VPN.

Run the following commands via SSH as root on the XCP-NG host:

sudo yum-config-manager --add-repo https://pkgs.tailscale.com/stable/centos/7/tailscale.repo
sudo yum -y install tailscale

and enable & start the tailscale daemon tailscaled:

systemctl enable --now tailscaled

 

Posted by Uli Köhler in Headscale, Networking, Virtualization, VPN

How to set X-Forwarded-Proto header in nginx

Directly after any proxy_pass line add

proxy_set_header X-Forwarded-Proto $scheme;

Typically X-Forwarded-Proto is used together with X-Forwarded-Host like this:

proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;

 

Posted by Uli Köhler in Networking, nginx, Wordpress

How to run iperf3 on Synology NAS using SSH

I prefer this method to the GUI docker method because:

  • It’s much more reproducible in practice
  • It involves fewer steps
  • It uses --net=host and therefore doesn’t involve additional routing, bridging or forwarding of packets which might impact performance

Login to the Synology NAS over SSH using a user with admin privileges, then sudo su.

For using iperf3 as a serve, use

docker run  -it --rm --name=iperf3-server --net=host networkstatic/iperf3 -s

For using iperf3 as a client, use

docker run  -it --rm --name=iperf3-client --net=host networkstatic/iperf3 -c 10.1.2.3

 

Posted by Uli Köhler in Networking

Real-world Tailscale iperf3 results between a VM and a bare metal Desktop on a switched network

We tested iperf3 performance using our network based on the following devices:

  • Desktop: Ubuntu 21.10 i7-6700 CPU @ 3.40 GHz, connected using 1Gbase-T to
  • Desktop switch: Mikrotik CSS610-8G-2S+IN connected using 10GBase-T multimode SFP+ module to:
  • Core switch: Mikrotik CRS309-1G-8S+IN, connected using 10GBase-T DAC cable to
  • Virtualization server: i5-6500 CPU @ 3.20GHz running XCP-NG 8.2.1
  • Virtual Machine: Ubuntu 20.04, 4 cores, 8GB RAM

Tailscale version was

1.24.1
  tailscale commit: 1a9302b1edba91d0f638e775faeaa0ce2a6a25f8
  other commit: 1331ed5836e1a0ab32b10e6ce8748e17ba2c7598
  go version: go1.18.1-ts710a0d8610

 

The network is completely switched, not routed and we took care that tailscale actually used the switched connection using tailscale ping.

Test 0: Direct connection over switched network

Desktop running iperf -s, VM running iperf -c 10.9.2.10:

Connecting to host 10.9.2.10, port 5201
[  5] local 10.9.2.103 port 52944 connected to 10.9.2.10 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  94.7 MBytes   794 Mbits/sec  338    109 KBytes       
[  5]   1.00-2.00   sec  98.0 MBytes   822 Mbits/sec  353    148 KBytes       
[  5]   2.00-3.00   sec  96.6 MBytes   811 Mbits/sec  382    117 KBytes       
[  5]   3.00-4.00   sec   103 MBytes   862 Mbits/sec  334    116 KBytes       
[  5]   4.00-5.00   sec   101 MBytes   851 Mbits/sec  483    102 KBytes       
[  5]   5.00-6.00   sec   104 MBytes   874 Mbits/sec  503    126 KBytes       
[  5]   6.00-7.00   sec   105 MBytes   883 Mbits/sec  527    119 KBytes       
[  5]   7.00-8.00   sec   108 MBytes   906 Mbits/sec  451    105 KBytes       
[  5]   8.00-9.00   sec   108 MBytes   903 Mbits/sec  442    117 KBytes       
[  5]   9.00-10.00  sec   107 MBytes   900 Mbits/sec  461    123 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.00 GBytes   861 Mbits/sec  4274             sender
[  5]   0.00-10.00  sec  1.00 GBytes   860 Mbits/sec                  receiver

iperf Done.

VM running iperf -s, Desktop running iperf -c 10.9.2.103

Connecting to host 10.9.2.103, port 5201
[  5] local 10.9.2.10 port 42630 connected to 10.9.2.103 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  88.5 MBytes   742 Mbits/sec    0    966 KBytes       
[  5]   1.00-2.00   sec  90.0 MBytes   755 Mbits/sec    0   1.12 MBytes       
[  5]   2.00-3.00   sec  87.5 MBytes   734 Mbits/sec   33    833 KBytes       
[  5]   3.00-4.00   sec  90.0 MBytes   755 Mbits/sec    0    833 KBytes       
[  5]   4.00-5.00   sec  88.8 MBytes   745 Mbits/sec    0   1.00 MBytes       
[  5]   5.00-6.00   sec  88.8 MBytes   744 Mbits/sec    0   1.00 MBytes       
[  5]   6.00-7.00   sec  87.5 MBytes   734 Mbits/sec    0   1.09 MBytes       
[  5]   7.00-8.00   sec  90.0 MBytes   755 Mbits/sec    0   1.09 MBytes       
[  5]   8.00-9.00   sec  90.0 MBytes   755 Mbits/sec    0   1.09 MBytes       
[  5]   9.00-10.00  sec  90.0 MBytes   755 Mbits/sec   13    863 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   891 MBytes   747 Mbits/sec   46             sender
[  5]   0.00-10.00  sec   888 MBytes   745 Mbits/sec                  receiver

iperf Done.

The direction where the VM hosts the iperf -s server i.e. sends the data shows a slight degradation of performance

Test 1: Desktop running iperf -s, VM running iperf -c

Connecting to host 100.64.0.2, port 5201
[  5] local 100.64.0.3 port 37466 connected to 100.64.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  39.4 MBytes   330 Mbits/sec   62    149 KBytes       
[  5]   1.00-2.00   sec  45.8 MBytes   385 Mbits/sec   44    150 KBytes       
[  5]   2.00-3.00   sec  38.9 MBytes   326 Mbits/sec   97    122 KBytes       
[  5]   3.00-4.00   sec  47.9 MBytes   401 Mbits/sec    7    242 KBytes       
[  5]   4.00-5.00   sec  39.5 MBytes   332 Mbits/sec  118    110 KBytes       
[  5]   5.00-6.00   sec  46.6 MBytes   391 Mbits/sec   32    136 KBytes       
[  5]   6.00-7.00   sec  41.8 MBytes   351 Mbits/sec   42    159 KBytes       
[  5]   7.00-8.00   sec  44.3 MBytes   372 Mbits/sec   91    104 KBytes       
[  5]   8.00-9.00   sec  36.1 MBytes   303 Mbits/sec   72    133 KBytes       
[  5]   9.00-10.00  sec  41.5 MBytes   348 Mbits/sec   39    139 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   422 MBytes   354 Mbits/sec  604             sender
[  5]   0.00-10.00  sec   421 MBytes   353 Mbits/sec                  receiver

iperf Done.

Test 2: VM running iperf -s, Desktop running iperf -c

Connecting to host 100.64.0.3, port 5201
[  5] local 100.64.0.2 port 36744 connected to 100.64.0.3 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  23.7 MBytes   199 Mbits/sec  104   89.9 KBytes       
[  5]   1.00-2.00   sec  23.6 MBytes   198 Mbits/sec   80   49.2 KBytes       
[  5]   2.00-3.00   sec  21.1 MBytes   177 Mbits/sec   59   54.0 KBytes       
[  5]   3.00-4.00   sec  23.6 MBytes   198 Mbits/sec   68   69.6 KBytes       
[  5]   4.00-5.00   sec  19.1 MBytes   160 Mbits/sec   77   48.0 KBytes       
[  5]   5.00-6.00   sec  25.3 MBytes   212 Mbits/sec   76   62.4 KBytes       
[  5]   6.00-7.00   sec  21.4 MBytes   179 Mbits/sec   50    107 KBytes       
[  5]   7.00-8.00   sec  25.6 MBytes   215 Mbits/sec   35    124 KBytes       
[  5]   8.00-9.00   sec  22.5 MBytes   188 Mbits/sec   71   48.0 KBytes       
[  5]   9.00-10.00  sec  25.0 MBytes   209 Mbits/sec   42   64.8 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   231 MBytes   194 Mbits/sec  662             sender
[  5]   0.00-10.01  sec   230 MBytes   193 Mbits/sec                  receiver

UDP tests

UDP tests were mostly similar to TCP tests (albeit slightly higher throughput at up to 400 Mbit/s), including the sensitivity to the direction of the connection.

Interpretation of the results

Tailscale has significant impact on network speeds and will not regularly be able to achieve near-Gigabit iperf3 speeds given typical setup with Desktop that are a couple of years old, and virtual machines. However, achieving a throughput of 200-400 Mbit/s is more than enough for most applications.

Interestingly, the speed is highly dependent on the direction of transfer between a less powerful VM and a more powerful Desktop, with a factor of x1.5 … x2 between the two directions. This might be attributed to the amount of computation required to encrypt or decrypt the data.

Posted by Uli Köhler in Networking

How to enable/disable WireGuard peer by comment on MikroTik

To enable the WireGuard peer called MyPeer:

/interface/wireguard/peers/enable [find comment="MyPeer"]

To disable the WireGuard peer called MyPeer:

/interface/wireguard/peers/disable [find comment="MyPeer"]

 

Posted by Uli Köhler in MikroTik, Networking

How to install tailscale on Fedora CoreOS

In order to install tailscale, on Fedora CoreOS (this post has been tested on Fedora CoreOS 35), you can use this sequence of commands:

sudo curl -o /etc/yum.repos.d/tailscale.repo https://pkgs.tailscale.com/stable/fedora/tailscale.repo
sudo rpm-ostree install tailscale

Now reboot using

sudo systemctl reboot

Once rebooted, you can enable the service using

sudo systemctl enable --now tailscaled

and then configure tailscale as usual:

sudo tailscale up --login-server .... --authkey ...

Also see our post on How to connect tailscale to headscale server on Linux

Posted by Uli Köhler in CoreOS, Headscale, VPN

How to start Jupyter Lab for remote access

This will start Jupyter listening on all network interfaces / bind to all IP addresses in order to make direct browser access possible not only from localhost but any remote host that has network access to the host where you’re running Jupyter:

jupyter lab --ip=0.0.0.0

 

Posted by Uli Köhler in Networking, Python

How to set DLS ContactMe request to OpenStage 40 phones using Python

This snippet sends a ContactMe DLS provisioning request to an OpenStage 40 phone at IP 192.168.178.245 using Python. The phone will then contact 192.168.178.10 on port 18443 using HTTPS. By default (i.e. if the OpenStage 40 is not in Secure Mode, the certificate is not verified – any certificate will do!)

import requests

response = requests.post("http://192.168.178.245:8085/contact_dls.html/ContactDLS", data={
    "ContactMe": True,
    "dls_ip_addr": "192.168.178.10",
    "dls_ip_port": 18443
})
# Response will be <Response [204]> i.e. no content

Note that dls_ip_addr may also be a hostname!

Posted by Uli Köhler in Networking, Python