{"id":4179,"date":"2021-03-02T21:02:49","date_gmt":"2021-03-02T21:02:49","guid":{"rendered":"https:\/\/www.piglet-net.net\/pigblog\/?p=4179"},"modified":"2021-03-04T12:41:05","modified_gmt":"2021-03-04T12:41:05","slug":"not-helpful","status":"publish","type":"post","link":"https:\/\/www.piglet-net.net\/pigblog\/?p=4179","title":{"rendered":"Not Helpful"},"content":{"rendered":"<p>The ongoing Covid situation means I&#8217;m still working at home, and now there&#8217;s demands for more colleagues to be able to work more normally from home too: a big hole in this was answering phones.<\/p>\n<p>I&#8217;ve worked with IP telephony a long time: since around 2004 in fact, and we&#8217;ve had the ability to do VoIP over a VPN for almost as long, so we&#8217;d provided phones to a small subset of staff in that way.<\/p>\n<p>To enable the same for non-VPN users, though, you need to somehow expose <a href=\"https:\/\/en.wikipedia.org\/wiki\/Session_Initiation_Protocol\">SIP<\/a> to the Internet. You could just do this direct, but SIP bots are a thing, so the solution is an <a href=\"https:\/\/en.wikipedia.org\/wiki\/Session_border_controller\">SBC<\/a>: this is essentially a firewall\/router\/application gateway that polices the incoming connections, and does things like rate limiting, user-agent monitoring, and DDoS protection, and hopefully stops the baddies getting at the PBX.<\/p>\n<p>All seemed straightforward: a VMWare appliance, a few strategic port-forwards at the ISP firewall, bit of config in the PBX, job done.<br \/>\n<!--more--><\/p>\n<p>So it should have been, as we were using people to install and configure the SBC, as at that point, I&#8217;d not had to troubleshoot SIP and had only vague ideas of what was involved, but hey, we&#8217;re paying experts here, huh? We had a list of ports to open, all looking good.<\/p>\n<p>Except, it didn&#8217;t work. Phones could register, and make calls, but no audio would pass.<\/p>\n<p>We were asked to get SIP Helper and SIP ALG disabled on the Fortinet Firewalls, and we did. At this point, things got a bit random: the PBX guys wanted more and more ports opening, we could see UDP packets hitting the SBC and nothing getting done with them. The SBC was rebuilt, more port opens were requested.<\/p>\n<p>[Screech of brakes]<\/p>\n<p>It&#8217;s now feeling like randomly changing things blindly and hoping it works. Time to read up on what SIP actually does.<\/p>\n<p>A SIP phone call, as nicely explained by Giampaolo Tucci <a href=\"https:\/\/www.informaticapressapochista.com\/asterisk\/from-sip-to-rtp-part-1\/\">here<\/a> and <a href=\"https:\/\/www.informaticapressapochista.com\/asterisk\/from-sip-to-rtp-part-2\/\">here<\/a> is very much two things: SIP actually does the call setup and control; the actual audio in the call is entirely separate and uses RTP, an application-layer protocol over UDP. The RTP can take a different path; in fact in some implementations, two phone handsets will talk to each other peer-to-peer, while the signalling traffic goes back and forth to the PBX.<\/p>\n<p>A key part is that when the call is setup, the two ends of the conversation send an instruction on what IP address and port to send audio on. This is why one-way speech or no speech is a common problem, and also why firewalls have so-called &#8220;helpers&#8221;, because if the two ends are unaware of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Network_address_translation\">NAT<\/a> taking place, the stream might go to the wrong place.<\/p>\n<p>We didn&#8217;t have that: we could see stuff arriving at the SBC, but being ignored: Our SBC should handle all that NAT-traversing stuff, so why is it ignoring the traffic?<\/p>\n<p>Part of the negotiation sends a SIP attribute:<\/p>\n<p><code>m=audio XXXX RTP\/AVP 8 0 96 98 99 97<\/code><\/p>\n<p>Where XXXX is the UDP port to send the stream to, and the other stuff is the details of the stream.<\/p>\n<p>Time for the last refuge of the desperate: packet captures. I&#8217;m fortunate here that my home router runs <a href=\"https:\/\/www.piglet-net.net\/pigblog\/?p=3187\">OpenWRT<\/a>, so you can capture there, and the SBC does that too.<\/p>\n<p>So, here is what we had:<\/p>\n<figure id=\"attachment_4180\" aria-describedby=\"caption-attachment-4180\" style=\"width: 300px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.piglet-net.net\/pigblog\/images\/\/network.jpeg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/www.piglet-net.net\/pigblog\/images\/\/network-300x152.jpeg\" alt=\"\" width=\"300\" height=\"152\" class=\"size-medium wp-image-4180\" srcset=\"https:\/\/www.piglet-net.net\/pigblog\/images\/network-300x152.jpeg 300w, https:\/\/www.piglet-net.net\/pigblog\/images\/network-1024x518.jpeg 1024w, https:\/\/www.piglet-net.net\/pigblog\/images\/network-768x389.jpeg 768w, https:\/\/www.piglet-net.net\/pigblog\/images\/network.jpeg 1280w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><figcaption id=\"caption-attachment-4180\" class=\"wp-caption-text\">A crappy network diagram, dig those l33t Visio skillz!<\/figcaption><\/figure>\n<p>Time to get the captures out and fire up <a href=\"https:\/\/www.wireshark.org\">Wireshark<\/a>, the kind of tool you&#8217;d have paid thousands for in the 90s, and not had the level of diagnosis. You can view the call setup conversation and see what the call setup is actually doing.<\/p>\n<p>Pulling up the traces side by side revealed the packets getting mangled en route: the port numbers were altered, the <code>m=audio<\/code> field was being altered too.<\/p>\n<p>Here&#8217;s the &#8220;same&#8221; packet, either end of the trace (the RH side being at my house, the LH at work). This is a part of the RTP stream, rather than the call setup.<\/p>\n<figure id=\"attachment_4182\" aria-describedby=\"caption-attachment-4182\" style=\"width: 300px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.piglet-net.net\/pigblog\/images\/\/Untitled-2.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/www.piglet-net.net\/pigblog\/images\/\/Untitled-2-300x169.png\" alt=\"\" width=\"300\" height=\"169\" class=\"size-medium wp-image-4182\" srcset=\"https:\/\/www.piglet-net.net\/pigblog\/images\/Untitled-2-300x169.png 300w, https:\/\/www.piglet-net.net\/pigblog\/images\/Untitled-2-1024x576.png 1024w, https:\/\/www.piglet-net.net\/pigblog\/images\/Untitled-2-768x432.png 768w, https:\/\/www.piglet-net.net\/pigblog\/images\/Untitled-2-1536x864.png 1536w, https:\/\/www.piglet-net.net\/pigblog\/images\/Untitled-2.png 1920w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><figcaption id=\"caption-attachment-4182\" class=\"wp-caption-text\">Packet capture showing packets altered in transit.<\/figcaption><\/figure>\n<p><strong>The packets are being altered!<\/strong> Not just address-wise, for NAT, but the actual data is being altered. in the capture at my house, you can capture the audio and play it, but the other end doesn&#8217;t even get recognised as RTP.<\/p>\n<p>Turns out that as well as SIP helpers, the Fortinet has a VoIP session-helper, which is trying and failing to help. It&#8217;s rewriting traffic in transit, transcoding ports instead of just passing it, like an older, simpler device would. Like so many things when computers try to be helpful, it&#8217;s confusing and gets in the way.<\/p>\n<p>The answer? Ignore the voice guys clamouring for more open ports, which wouldn&#8217;t have worked and exposed more of the network. Present the ISP with the evidence, including full packet captures, and let them discover the additional helper and turn the damned thing off \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The ongoing Covid situation means I&#8217;m still working at home, and now there&#8217;s demands for more colleagues to be able to work more normally from home too: a big hole in this was answering phones. I&#8217;ve worked with IP telephony a long time: since around 2004 in fact, and we&#8217;ve had the ability to do &hellip; <a href=\"https:\/\/www.piglet-net.net\/pigblog\/?p=4179\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Not Helpful<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-4179","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=\/wp\/v2\/posts\/4179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4179"}],"version-history":[{"count":0,"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=\/wp\/v2\/posts\/4179\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4179"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4179"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.piglet-net.net\/pigblog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}