Router EfficienciesISP Routing Success through MikroTikBest Practice Implementation

visp.net/blog

Joshaven PotterMay 21, 2019

MikroTik is a software-defined firewall and router. As such, it’s limited not only by the hardware it runs on but also by how it is configured. Just like a computer, running too many processes can overburden it, dragging performance down and causing network issues.

When a packet passes through your router, it

passes through a variety of facilities including

NAT, Mangle, Firewall and Bridging. This can

happen hundreds of thousands of times per

second. Since each facility typically has many

rules, a packet traversing this digital maze may

be touched numerous times. Issues can quickly

compound with poorly configured routers

causing resource exhaustion, which frustrates

both you and your customers.

When you build your network, remember…

packets are people, just like you and me.

Sometimes packets are Facetiming with

Grandma, holding your position in a first-

person shooter game or stopping an attempted

infiltration from a hostile hacker trying to

acquire digital assets for a cyber war.

If you are going to undertake the responsibility

of improving the efficiencies of your routers,

then I highly recommend you read this MikroTik

best practices overview and then spend some

time deep in the study of exciting things like

MikroTik’s packet flow diagrams.

THE BIG PICTURE

visp.net/blog 2

visp.net/blog 3

Less is better, almost always. Asking your router to do extra work on every packet processed is likely to create problems at some point. Finding ways to process your packets with shortcuts can reduce processing loads.

One of my WISP clients had VoIP quality issues which turned out to be caused by an overused address-list lookup. The surprising part was that the router was among the most powerful, a Cloud Core Router running no higher than a couple of percent of CPU utilization.

We were able to determine that the address-list was being queried tens of thousands of times a second. The trick that lead to finding this overused address list was preceding the rule with a counter rule (without the address-list condition), so we could track how many packets per second were being processed. After removing the need to check the address list so often the VoIP quality was fixed.

I often see simple things being skipped such as accepting established and related traffic and no use of connection fasttrack or fastpath where possible.

One of the “gotchas” with fasttrack to be aware of is that it will shortcut your router’s ability to account for data skipping router features like simple queues. This can, for example, affect your billing system’s ability to calculate usage. You can’t use it everywhere but it helps a lot with efficiency where you can use it.

Don’t forget that order matters so the biggest bang for your buck rules should be as high in the rule list as possible.

MINIMALIZE YOUR ROUTER PROCESSES

Your router and your network will be harassed by hackers and bots, so make sure you have rules that guard your gear. Best-practices include keeping your equipment on a protected management VLAN, protecting access to management networks and keeping software patched and updated. Permit only required traffic and block all else in your router’s input firewall filter chain.

One trick to implementing this in a live network is adding a log rule then watch the logs to catch anything you might have forgotten to add. Before you enable the “drop all” rule in your firewall filter input chain, turn on Safe Mode just in case. Don’t forget that if you are accepting established and related traffic you may not see existing connections from the log rule.

SECURE YOUR EQUIPMENT

visp.net/blog 4

MONITORING WITH NOTIFICATIONSOne of the biggest differences I see between small and large networks is their monitoring. This is because you will reach a ceiling as an organization without a good monitoring system. It might kinda feel like I just switched topics, but honestly, the efficiency of your routing system is directly affected by how you watch it.

Watched metrics, as a rule, improve within an organization because you are able to see things like climbing CPU, interface errors, temperature alerts and many other metrics that indicate your system’s health. This can allow you to be proactive and not reactive when it comes to issues on your network.

STANDARDIZATION, CONFIGURATION BACKUP AND MANAGEMENTThe configuration of similar systems should be the same, excepting only the elements that need to be different given the physical differences at the site. This is easy to say, easy to understand, but hard to consistently implement.

If a high level of consistency is achieved, then a senior network administrator should be able to rebuild a config on the fly with limited information… this isn’t a best practice, but it is a good test of how clean, logical and consistent your configuration is within your network.

Keep current configuration backups and spare hardware handy. The last time a client had damaged hardware was yesterday, literally… it happens, it shouldn’t be the norm but, as a service provider, you need to be prepared.

visp.net/blog 5

COMMON MISTAKESFirst and foremost, make sure that you not only have on-staff expertise but surround yourself with friends, co-workers and support partners who have a genuine interest in your success.

Maybe it should go without saying but looking at what your router is doing is one of the main ways to improve efficiency. If you have not mastered the regular use of the primary efficiency tools, then you aren’t very likely to have an efficient router setup. You can start by mastering the use and inspection of: 1) tools profile, 2) /interfaces, 3) /tools torch, 4) / system profile and 5) your log files!

A common culprit of CPU issues is the use of the masquerade action, especially with PPPoE concentrators. Yep, this common action should be avoided when possible. Every time an interface changes state or IP’s, the masquerade action will cause the router to have to reconcile all of its connections, purging anything related to that interface. An easy solution is to use src-nat instead of masquerade. Use masquerade when the source IP can dynamically change, just don’t overuse it.

Operate on connections rather than packets. Make sure that you are not rechecking connections over and over again, especially with expensive tasks. You can murder the CPU on even high-end routers with a few poorly positioned, expensive rules. Once connections are marked, you can mark packets if needed.

Take, for example, blocking Facebook access with a layer 7 rule. Adding connection marks on new DNS lookups for Facebook can allow your router to only have to look deeply into the packet for the DNS request and then track and block all the communication related to that lookup thereafter.

Unimus is a software solution which helps network administrators implement and improve upon every best practice mentioned in this article in one way or another.

Unimus excels in the area of backup and configuration management. It simplifies tasks like comparing configurations between devices, auditing your network for best practice implementations, and firmware and configurations deployment.

“Automating configuration management can save you a lot of time (and therefore a lot of money). Unimus for example can help with automating RouterOS upgrades on your MikroTiks, and manage the overall consistency of your network. With recent MikroTik exploits, you can easily check the health of your entire network in bulk, and automate the remediation of infected routers.” — Tomas Kirnak, Founder / CEO at NetCore (Unimus)

NOTEWORTHY MENTION

visp.net/blog 6

UNCOMMON BUT DISASTROUS MISTAKES

It is a classic issue that is uncommon due to how fast exploits wreak havoc,

however, don’t ever enable remote requests for DNS unless rejecting DNS queries

from everyone except the hosts that you should be answering.

When using simple queues, use targets. When you don’t specify a target, your

target is all traffic which means all packets must be processed by the queue. If you

are using the dst option to specify your targets, then you are asking your router to

do a bunch of work for no reward. One simple queue with a target of 0.0.0.0/0 can

break your queues, making it capture all of your traffic.

If you have high CPU loads on your PPPoE server, you might be spamming your

routers with /32 route updates. Unless required, avoid having PPPoE routes in OSPF.

You don’t want to recalculate your routes every time a user’s status changes. This

can be accomplished by excluding the PPPoE interface from OSPF with the passive

flag. If you must include PPPoE routes in OSPF, then be sure to use stub areas to

reduce the amount of information flooding through OSPF. This can cause customer

speed issues and it can snowball from having overload conditions cause more

PPPoE disconnects.

Practical steps for building good configurations

• Understand chain, packet flow and how to get diagnostic data out of your router.

• Use Smart Filter Rules.

• Filter rules should be written in the proper order which means taking shortcuts, using fast tracks and working with the connecting tracking engine.

• Always accept established and related traffic before rechecking already accepted traffic.

• Where possible, use the RAW table to bypass connection tracking.

• Utilize connections for efficient packet processing whenever possible.

• Avoid overusing the address-lists as it is often more expensive than alternative solutions.

• Use src-nat instead of masquerade whenever possible.

• Don’t spam OSPF with rapid changes.

• Limit access to your router to include only the traffic that is required.

• Leave customer access open except where it makes your network vulnerable.

You can begin with an all-in-one core router but as you grow you will likely want to segment

visp.net/blog 7

You may want to block some traffic at your edge that is a potential

threat to your network, like DNS & Time which can be used for

amplification attacks, or doesn’t belong on the Internet, like Windows

local file sharing. Below is a non-exhaustive table of services that

you may want to block as a standard policy for new connections on

network ingress. You can always make source-based exceptions.

20/tcp FTP data connection

21/tcp FTP control connection

23/tcp Telnet protocol

25/tcpSimple Mail Transfer Protocol (SMTP), used for email routing

between mail servers

53/tcp DNS

53/udp DNS

67/udp Bootstrap protocol or DHCP Server

68/udp Bootstrap protocol or DHCP Client

123/udp Network Time Protocol (NTP)

135/tcp/udp

Microsoft EPMAP (End Point Mapper), also known as DCE/RPC

Locator service,[65] used to remotely manage services including

DHCP server, DNS server and WINS. Also used by DCOM

137/tcp/udp NetBIOS Name Service, used for name registration and resolution

138/tcp/udp NetBIOS Datagram Service

139/tcp/udp NetBIOS Session Service

161/udp Simple Network Management Protocol (SNMP)

179/tcp Border Gateway Protocol (BGP)

1080/tcp SOCKS proxy protocol

8291/tcp MikroTik Winbox

8728/tcp MikroTik API

8729/tcp MikroTik SSL-API

About the Author

Joshaven Potter supports and consults

with Visp.net’s clients to drive their success and

business growth. He is an industry-leading WISP

consultant with 18 years of ISP network experience.

Want to read more articles that focus on your profitability?

Check out the Wisp Success Blog at https://visp.net/blog

When you’re ready to ignite your growth, choose a billing vendor with a commitment to WISP success.

YOUR SUCCESS ISN’T OPTIONAL

With ULTIMATE BACK OFFICE BILLING AND AUTOMATION, WISPs streamline the acquisition of new subscribers, better control the customer experience and grow revenue.

When you celebrate your WIN, we all WIN.

visp.net541-955-6900

When your Billing and Automation system links subscriber access to their billing, your business becomes a revenue machine that feeds your profit. You will:

Speed up new subscriber on-boarding

Always get paid for the bandwidth you provide

Deploy Hotspots anywhere your network reaches