The question of escalated listings is not a simple one to many people, but escalated listings in reality are simple and are there for a simple reason.

First as per the title of this document a little explanation is in order. Many people confuse escalated listings with single IP multi server hosts (aka Virtual server hosts). The differences between the two are very simple.

A multi server host (aka virtual server host) has one or more IP addresses and lots of virtual hosts. The virtual server hosts can be full virtual machines or can be just virtual webservers, the difference is not important here as the net effect is a single piece of hardware uses a single IP address for multiple clients (people/customers). When the IP address sends spam, it is listed at places like SORBS and mail from that host (for all the servers) is blocked or marked as spam.

An escalated listing on the other hand is where a whole network of IP addresses i listed in SORBS and all hosts and IPs (whether assigned to a single customer or multiple) are listed and therefore blocked or result in spam folder issues.

Why does SORBS create escalated listings?

The simple answer is to stop spam.

You ask, “How does listing innocent IPs help stop spam?”

Simple, some providers don’t care about spam, to them the bottom line is all that counts. Spammers pay lots of money for servers and as such the company doesn’t want to kick them off their servers. As an exmple one of the SORBS servers in the USA is a 16 core server with 3T of monthly traffic, which we pay around $1700/month for. $1700/mo is no insignificant amount and if you are a sales person with a monthly sales target of $10k per month (ie if you don’t hit that minimum every month you get fired) and you have a spammer that rents 3 of these servers, you don’t need a lot more servers to hit your monthly target. Better yet most sales people get commission for sales over their minimum, so the more of these servers you sell the more you make. Bear in mind at this point a normal web service host that you might purchase is usually around $100/mo… A significant difference!

So why list multiple hosts, well in the example above a sales person making even just $1700 per month from his spamming customer is not going to terminate (or even chastise) a customer that is causing abuse reports. He’s not going to care if the SORBS admins add the IP address the spammer is using to the SORBS lists. What he will care about is if his other customers suddenly start complaining, he’ll be even more likely to terminate the spammer if he starts losing money. So thinking about the maths, you have one spammer paying in $1700/mo and 20 customers paying in $100/mo, you’re going to ignore the issue unless 17 of your customers decide to go somewhere else…

Now does SORBS like this idea? No of course not, we’d prefer to send in the abuse report and see the spammer removed from the network post-haste. We don’t want to list innocent people, and so we use it as a last resort. Our Spam DB FAQ details about escalated listings and how we do it, we rarely follow it to the strict letter of the policy, mostly allowing a lot longer time limits and more spam before we escalate. For known spammers we follow the policy strictly being very forceful as quick as possible.

SORBS’ goal is to stop spam, not to make money, and not to help anyone else make money. The simple fact is if you are using the same server as a spammer you will be blocked, if you are in the same network as a spammer you will be blocked. If you are using the same ISP as the spammer and the ISP chooses to continue hosting the spammer and ignore SORBS you will be blocked, and in the latter case we recommend you go find another ISP.

So what’s the chances of you moving from one ISP to another ISP with spammer problems?

Well that question depends on how much you are spending. If you go looking for the cheapest servers, the chances are you’ll find the servers where spammers have already been or where they are still. The best thing you can do is talk to a sales person and ask them about SORBS listings, ask them what would happen if you get listed. If he says, “oh don’t worry about it we’ll help you sort it”, or “whilst your paying your monthly fee, we don’t care” or some other variant, find another ISP, they are the providers that will cause you trouble in the long run. If on the other hand they say, “well we’ll terminate your contract on the first sign of trouble due to our strict AUP, and we might even charge you a cleanup fee”… Choose that ISP, the chances are they’ll never experience an escalated listing issue!!!

This is often a point of contention with people attempting (and especially after being refused or restricted on) delisting.

The SORBS staff do not participate in many email lists, some we are there to monitor, some we will talk about general issues, others we do not participate in.

The lists we participate in are:

• Zorch – a private and restricted membership list that participants have to be nominated then invited to. Spam and Email deliverability is discussed.
• NANOG – an open list where network issues are discussed.
• NZNOG – an open list where network issue are discussed.
• UKNOF – an open list where network issues are discussed.
• ASRG – an open list where spam blocklists are discussed.
• Postfix-Users: an open list where the postfix mail server is discussed.
• Patternity – where spam netblocks are reported.
• dnsbl-users – an open list where the usage of SORBS can be discussed.

Lists which SORBS admins may or may not be monitoring but will never reply to a message:

• Spam-L – an open list created to replace the original Spam-L that is to discuss spam.

There are also various forums and news groups (eg: news.admin.net-abuse.email – NANAE) where SORBS staff sometimes post.

SORBS Support issues are discussed in some of the forums, mailing lists, and news groups, but regardless of SORBS membership will never be resolved by discussion on such a group. A good example of this is the Spam-L list where the main SORBS admin does not participate or subscribe following a decision of the moderation team to get involved in an argument about the merits or problems of SORBS then disable the ability of SORBS staff to reply. Another example is Zorch where SORBS staff removed themselves from the list when it was found that private information discussed was repeated by some members in public forums such as Spam-L. The policy of Zorch has been and should always be that discussions on the list are not to be repeated off the list except where express (written) permission was given, the vetting of members is designed to aid in that policy, but fails from time to time due to accidental (sometimes deliberate) disclosure.

In all cases listing and delisting of IPs or networks must be discussed and approved in the SORBS Support system ( http://support.sorbs.net/ ) anyone offering advice to the contrary is not a SORBS staff member or will be removed from the staff very shortly there after.

For this reason it is a pointless exercise and waste of time to discuss these issues there.

It has come to SORBS’ attention that some providers of mail services are sending out messages similar to MessageLab’s message:

Thank you for contacting SHS Global Client Support regarding blocks
   on our email infrastructure.
 
There are many organisations/authorities around the world that offer
    a means of blocking email based on list entries that their users
    can subscribe to.  Whilst many are entirely legitimate and
    beneficial, some authorities have aggressive listing policies or
    lists that are almost impossible to be removed from.  As a result,
    SHS do not recognise these authorities as legitimate.
 
Some of the block list providers will charge listed parties to
    become removed from the list.  Once the fee has been paid and the
    IP delisted, the owner of the IP may find that they have become
    listed again a very short time later.  Lists like this aren’t
    providing a beneficial service and their use can cause mails not to
    be delivered even if the mails/sender are legitimate.
 
Other block list providers that we do not recognise as legitimate
    may look at the way email is handled.  Some block lists will spot
    the way our service relays mail from client to recipient and from
    sender to client.  Our relay-based service means that our
    infrastructure can become blocked causing unnecessary mail
    failures.
 
In instances where SHS have become blocked by what we consider to be
    “unhelpful” block list providers, we are unable to request de-
    listing of our IP addresses.  In these instances, we ask the client
    in question if they would kindly notify the intended recipient that
    they are using a block list that causes large amounts of false
    positives and that email from yourselves has been stopped.  The
    intended recipient should be discouraged from using such lists in
    favour of more legitimate lists available on the internet.
 
We are currently aware of some block list providers that can cause
    problems with SHS email routing and that we are unable to resolve
    by becoming de-listed.  These lists are known as:
 
 -          SORBS (www.sorbs.net)
 -          BACKSCATTERER (www.backscatterer.org)
 -          SPEWS (APEWS)
 
 SORBS will list very aggressively and often without consideration to
    the services we are offering our clients.  To become removed from
    the SORBS list, they charge a fee which will often be only a
    temporary de-listing before we get placed back on the list.  We
    have tried to work with SORBS but they are less than happy to co-
    operate with us despite our efforts against the constant threat of
    email borne malware and spam.  As a result we do not recognise them
    as a legitimate list provider and request any clients experiencing
    issues contact their intended recipients to advise of the problems
    caused by using the list.
 
BACKSCATTERER is a list that almost always contains SHS IP
    addresses.  As stated on the backscatterer.org website, their list
    is not one that contains spam preventing listings but instead
    sender callout abusers or users that send backscatter.  They advise
    on the site that many big email providers/ISP’s will send NDR
    (bounce messages) because the systems they use incorporate
    relaying.  Backscatter, is the process of sending NDRs to non-local
    users. Because of the way the SHS system operates, we have the need
    to deliver NDRs outside of our own network, or our clients would
    never be aware of messages that we were unable to deliver.  This
    puts us at odds with backscatterer.org.  If you are having an issue
    delivering to a third party who is using backscatterer as a spam
    list, the third party should be educated and instructed to further
    investigate the http://www.backscatterer.org/
    website to familiarize themselve
    s with the concept of the list. Backscatter's policy is that they
    maintain their own list for specific purposes and as such will not
    delist anyone, unless a fee is paid. Notably, they also recommend
    that their list is not used as a basis for rejecting mail.
 
SPEWS (APEWS) is a list that is no longer maintained.  It was once
    responsible for blocking large areas of “netspace” and only dealing
    with ISP’s.  Because the list is no longer in service, there may be
    DNS records that refer to an old database.  For this reason this
    list should not be used and de-listing is not a possibility.
 
There are also likely to be other organisations that operate in
    similar ways.  At present we have not discovered these lists but
    this response could stand for lists not named above.
 
Due to the reasons above, we are unable to become de-listed and as
    such unable to affect the course of any email destined to
    recipients using these lists.  Whilst we applaud third parties
    willingness to prevent spam and utilise such lists, there are many
    other more legitimate lists that can be utilised.  SHS work closely
    with those offering legitimate lists to ensure our infrastructure
    isn’t impacted should somebody abuse/compromise our system or one
    of our clients systems.
 
Please feel free to respond with any questions or comments regarding
    the above.  Alternatively if you wish to check the status of an
    IP/block list, please do not hesitate to contact us.
 
Once again, please accept our sincerest apologies for not being able
    to offer more assistance on this matter.
 
 
Kind Regards
 
*** *******
Support Centre Analyst
Symantec Hosted Services
www.messagelabs.com
   24x7 Global Client Support Centre:
US/Canada: +1 (866) 807 ****
EMEA: +44 (0) 870 850 ****
Australia: 1 (800) ******
Hong Kong: 1 (800) ******
Asia Pacific: +852 6902 ****
*****@messagelabs.com

One should note the immediate and glaring accuracy issue SPEWS isn’t, never was and never will be associated with APEWS (or ASPEWS – another replacement for SPEWS).

Now on to the message, the email is quite patently designed to mis-lead users of SORBS into believing that MessageLabs are the good guys and SORBS are the scum of the earth. The reality of the fact is that MessageLabs have refused to officially enter into negotiations with SORBS on any level on multiple occasions. They log support requests to ‘get delisted’ and sometimes are refused based on recent examples of spam or the sheer volume requiring a donation to a charity because of the spam volume, MessageLabs rarely (if ever) reply to our messages to continue negotiation. The SORBS support system is there to provide a recorded two way communication path, some, such as MessageLabs use it as a write only path. They write their messages and ignore any replies.

SORBS lists spamming IP addresses (sources of spam) that may be used for blocking of spam into the mail system that is utilising SORBS as a blocklist. MessageLabs have never paid for delisting from SORBS despite multiple removals from the SORBS spam database (please do read the article about the fine, SORBS only places the fine on sources of spam that keep spamming, and only for those sources that spam SORBS directly.) SORBS consists of multiple databases and does not charge for delisting from majority of the databases.

MessageLabs keep getting relisted in the SORBS database because they are contracted by their customers to deliver all messages they send. This appears to be the basis of their ‘right’ to send spam. SORBS disagrees, spam is spam, and if you as a provider do not implement effective outgoing spam filtering, you will be restricted on delivery options at some time.

SORBS is considered to be an aggressive blocklist but considering that SORBS receives more than 30 billion lookups per day by tens of thousands of users it is patently obvious that SORBS is a legitimate list regardless of MessageLabs (or any other company’s) claims.

The facts are simple, if you deliver spam to our servers you will be listed sooner or later. If you do not deliver spam to our servers you should never be listed. MessageLabs despite their claims to the contrary deliver spams to our servers on a regular and ongoing basis, and as such they are listed in our database of deliverers of spam (the SORBS Spam DB.)

UPDATE [24/Aug/2010]: SORBS has spoken to MessageLabs and the message being sent is now being changed, and we are discussing issues.  This page is being left as an example only as we see other messages similar to this from time to time from other providers.  Kudos to the admin @ MessageLabs that enacted the change and a discussion.

Another common point of contention over SORBS’ policies is the perceived TTL requirement.

The TTL issue is only a requirement on PTR record for Single IP delistings, and even then it is only a requirement for delisting from the SORBS DUHL database.

So what is a TTL and why is it an issue?

TTL means ‘Time To Live’ and is the time a record is held as valid and cached before rechecking.  TTLs apply to many things such as Web Pages and DNS records, where web page TTLs are used to tell your browser not to reload the page from the server for a minimum amount of time.  Browsers can ignore this value, but it is recommended that they do not.  In DNS it means when you enter “http://www.sorbs.net/” the computer knows to lookup www.sorbs.net, contact the host on port 80 and request a web page called www.sorbs.net.  In technical terms this is done in the following steps completely transparently to the user:

First the browser looks at the protocol requested (in this case http) and knows it is a HTTP (web) request, as there is no port specified it knows to use the default port of 80.

Next it grabs the hostname and looks up www.sorbs.net by sending the request to the local DNS server(s) using “Give me the A record for www.sorbs.net”. The DNS server will then consult it’s authoritative records, and if it’s not a SORBS name server will not find it.  The server will then consult it’s “Cache” entries and if it find an entry check the TTL.  The TTL will consist of two parts, the value and the time it was retrieved last, if the value added to the time retrieved is in the past the server will expunge the record and check it’s “root zone” file.  If the TTL+retrieval time is in the future it returns the record to host which in turn gives it to the browser.

The ‘root zone’ file is a list of ‘hints’, hints are a static list of where to go to find answers to the answers, in this case the hints will return the list of root servers which when queried will return NS records for the domain ‘sorbs.net’, they will also send the IP addresses associated with the names in the NS records.  These NS records can then be used to locate the servers with the authoritative information for the domain ‘sorbs.net’.  The resolving server (the local one to the browser) will then ask the authoritative servers for ‘sorbs.net’ for the A record for ‘www.sorbs.net’, the result, the time it was retrieved and the TTL of the record will then be added to the local ‘cache’ to save going through the process again for any other requests for the same information within the same time.  The resolving server then sends the information back to the host with the browser which in turn passes it to the browser.

At this point the browser has the request for ‘http://www.sorbs.net/’ and a DNS record that says ‘www.sorbs.net is an A record with value 111.125.160.134′ and will remember the record for the next 600 seconds (600 seconds being the TTL of www.sorbs.net).  The browser will then use the record to make an HTTP request from the server 111.125.160.134 on port 80 for the web page / on host www.sorbs.net.  The server sends the result and the browser will display the result to the user.

Nice and simple you might think, and so nice that it’s all done completely transparently in the background so you don’t have to know what is going on.  So why is there an issue with the TTL?  Well there are many records in DNS, not just the ‘www’ records, some will tell you what the hostname of the machine is, some will tell you where to send email to, and as we have already seen, some will tell you where you can get more information from.  TTLs just tell you how long that information is valid for.

SORBS uses this information to approve or deny requests, and has on a number of occasions had people try to fake the information for malicious purposes (eg Spammers getting delisted then faking the information to divert attention to someone innocent when they send spam)  For this reason we mandated that if we get requests for delisting of single IPs from the SORBS database we would require some minimum requirements, that being the information is valid for at least 12 hours and preferred 24 hours.  We believe that it should be valid for longer, but operationally this doesn’t make sense.  If a host is part of a cluster of servers and one of those servers has a problem, the administrators don’t want 100′s of people remembering where it is for hours at a time, they want all the requests to go to the remaining servers, and for this reason we don’t require TTLs to be any minimum value on the MX (Mail eXchanger) or A (Address) records, on the PTR (reverse PoinTeR) records.

Now why can the PTR records be high without them being a problem, well consider the following setup for SORBS.

SORBS has 7 mail servers in it’s data centers. 4 of these are ‘Mail eXchanger’ (MX) servers, 2 of them are higher priority than the others and they are as follows:

desperado.sorbs.net priority 10

scorpion.sorbs.net priority 10

catapillar.sorbs.net priority 5

anaconda.sorbs.net priority 5

The lower the number the higher the priority, and therefore all email for sorbs.net should get sent to catapillar.sorbs.net and anaconda.sorbs.net, only getting sent to scorpion.sorbs.net and desperado.sorbs.net if the first to are too busy to answer requests.  Now MX records give a list of hosts where the servers handle email for SORBS.net are, and those hosts when looked up will provide IP addresses (in the same way that the browser will request a webpage as described above.)

If anacoda.sorbs.net were to suffer a real problem (eg the power supplied fried and caught fire taking out the whole host) the remote servers would try it every second time, time out and then retry with catapilla, if catapilla is too busy it’ll fallback to either desperado or scorpion.  This means mail might be delayed so we would probably want to update the server list to exclude anaconda from the list, and if we have set the TTL to 86400 it means some servers will remember that anaconda is one of the server for upto 1 day.  Setting it longer results in a bigger delay for the change.  This is of course undesirable and is often used as the excuse for not changing the TTLs to the length SORBS requires for single IP delistings.

The argument is flawed.

SORBS requests only the PTR records be set to 86400 seconds (1 day) and not the A or MX records, so if you want to move the servers, or reconfigure because of outage, or other issue, you can do what ever you want.  The PTR record is the reverse PoinTeR record that translates the IP address back to a hostname.  The PTR record is used when your host contacts one of SORBS servers if you try to send email to it.  It is also used (for records) when you request a webpage, or when you register on SORBS.  How is it used you might ask..?  Well in most cases it’s just recorded to prevent/identify abuse, in other cases it might be used to block or allow access.  So as you can see it is not used for anything that would affect you operationally (unless you are trying to abuse something) so therefore the TTL should not really matter.  In fact the only time wehave ever seen a TTL matter is when a mistake is made, and the operator/admin wishes to correct the mistake and hosts remember the mistake for hours/days.

Many people use the “but I need to migrate my networks” excuse to try to invalidate the policy.  There are two reasons why this is not valid:

• First, you don’t migrate every day, and when you do migrate you should be planning it well in advance so you change the TTLs on everything to smaller and smaller ones as the migration approaches.
• Second, remember the PTR record is only used when your server contacts our server(s) so if you move that server to another IP the new IP’s PTR record should be setup accordingly (and in advance) then when the migration takes place the old and disused IP will retain the PTR record in caches for the length of the TTL (usually 1 day) but as there is no server on it, it’ll never be seen or recorded by anyone.

Another common argument against the policy is with ISPs where they say ‘but our customers might be disconnected and another customer allocated their IP address’

Good network management would result in a customer leaving and the IP addresses going into the ‘unallocated’ pool at the back of the list so until all the addresses have been cycled it is not re-used.  There are no ISPs that we know of that have a turn over of customers on their static IP addresses where all the available space is rotated within 24 hours.

There are many other arguments for and against the policy but we of SORBS have considered each one and can find no valid reasons to have PTR records of 60 seconds except where it comes to abuse, where someone will fake information, do something bad and then change it back.  We therefore mandate that if you want to remove a single IP address from the SORBS DUHL, the PTR record has to have a minimum TTL of 12 hours (43200 seconds.)

Note: Throughout this document we refer to a single IP address, this is because networks of 256 addresses (/24) or larger do not have the same policy as it is likely the ISP is performing a ‘mass update’ of data.  Should anyone request delisting from the SORBS database, the support robot looks at the PTR records for all addresses in the request and will reject the request if the TTLs are lower than 43200 seconds (even if it’s a network request.)   This is because the robot is simple, and is only for sorting the rubbish from the good requests.  Any ISP representative requesting a network delisting would need to reply to the robot response which will result in the request automatically being given to a SORBS administrator (a real human being) who can use a lot more complex logic to analyse and formulate a response to the request.

A continual source of complaints by the uneducated is that SORBS is trying to tell people how to run their networks.  Unfortunately there is not much we can say except that they are wrong on so many levels.  Here is some information about why.

First we believe that we don’t want email from networks that can’t follow basic network setup of services.  If you, as a network administrator, cannot be bothered to setup your network in accordance with good/best current practices we can be fairly sure you don’t care about basic network hygiene and we would probably see very legitimate traffic from your site as apposed to the masses of spam and other nasty traffic.

So what does best current practices really mean?

Well first basic setup for a network is getting routed, and if you get that wrong, well you’re not going to get any traffic so that’s not really an issue.

Next you’d give names to everything on your network.  Now this is where the first issue starts.  There are these things called ‘A’ records, ‘MX’ records, and ‘NS’ records (to name the most important few.)

NS records provide the “glue” for DNS (Domain Name System, the service that translates www.yourdomain.example.com into an IP address)  The “glue” tells the system where the authoritive names are for your domain(s) and without it anyone requesting a lookup would not be able to work out where to send the request.

A records provide the name to IP address translation.  Each A record will have a hostname which when combined with the domain name (using NS records) will return IP addresses of computer/server/host/cluster that you want to talk to.  The IP addresses are used for the computers to talk to each other, not the names.

MX records are “Mail eXchanger” records, which means a record which tell your computer where the server is that will receive email for the domain you are trying to email to.

This all seems quite simple, and most people get this right as there are many examples of how to setup your own domain on the internet.  The problem comes with the other records that people ignore such as the ‘PTR’ records (reverse “PoinTeR” records) in most cases PTR records are not required to make things work so people ignore them.  The PTR record though is used by remote machines (such as email servers) to verify who is sending the email and provide a human readable view.  What this means is when a computer has the IP address “1.2.3.4″ (which really means nothing) a mail server when receiving a connection from it will check for a PTR record and if it gets one (eg: mail.example.com) will insert that information into the message so anyone with a filter that says “I want email from example.com servers” will be able to whitelist it.  Seems simple enough so far, however some people also use it to base a complex set of rules and will usually check the PTR records match the A records and visa-versa (to ensure no forgery) and will reject all mail when this fails.  They may also (like AOL) reject all mail from IP addresses that don’t employ PTR records.

Best Current Practices (and the same recommended practice since the early 1990′s) are that every host has an A record and the IP address used in the A record has a matching PTR record.  Of course with later developments of the internet IP address space started to fill up and so you would find that one server might have many A records from differing domains.  RFC1912 was created in 1996 to address common DNS issues and addresses the PTR record from a BCP point of view.  A records do not have to have a matching PTR record but all PTR records MUST have a matching A record (Section 2.1) here is the quote:

Make sure your PTR and A records match. For every IP address, there should be a matching PTR record in the in-addr.arpa domain. If a host is multi-homed, (more than one IP address) make sure that all IP addresses have a corresponding PTR record (not just the first one). Failure to have matching PTR and A records can cause loss of Internet services similar to not being registered in the DNS at all. Also, PTR records must point back to a valid A record, not a alias defined by a CNAME. It is highly recommended that you use some software which automates this checking, or generate your DNS data from a database which automatically creates consistent data.

The PTR issue existing often is the source of support calls to ISPs, and as they want a quite time, they take the lazy option and create PTR records that look like ’1.2.3.4.example.com’ or ’1.2.3.4.dsl.example.com’ and then create a bunch of associated A records so the “forward-reverse’ verification doesn’t fail.  Being the ‘lazy’ option they also make the process as simple as possible, some more advanced admins are familiar with ‘$GENERATE‘ lines some just write scripts to populate the zone files statically.  The ‘$GENERATE‘ lines are not as silly an idea as all think though as they can be overrided.

To use the $GENERATE line you can add the following to a zone file:

$GENERATE 11-254 $ PTR dhcp$.example.com.

This will generate the following records:

11 PTR dhcp11.example.com.

12 PTR dhcp12.example.com.

13 PTR dhcp13.example.com.

…

253 PTR dhcp253.example.com.

254 PTR dhcp254.example.com.

Note: $GENERATE can be used for both PTR and A records so you can also use them to add the A records to match the PTR records.

So were does SORBS come into all this, well if SORBS see records for 65,000 consecutive addresses we feel that the admins of the network can’t be bothered with that part of their network, or that they feel everything in that area are non-consequential hosts (eg: home users that are browsing the Internet – and therefore not running servers.)  The result is that we can block those hosts as they either aren’t or shouldn’t be running email servers.

We believe all hosts on the Internet that are servers should be registered in the Domain Name System (DNS), we believe that for hosts that have multiple services (eg a web server hosting 100′s of domains) the servers’ hostname should be registered with both A and matching PTR records.  However the web services require the A records, but the PTR records are not mandated so should be skipped.  This is best current practices and everyone who follows these practices never run into issues with SORBS.

If you are an administrator that doesn’t follow the best current practices and has problems with SORBS now you know why…

Now why was the $GENERATE information given, well there are some ISPs who wish to set “default”, a set of PTR records that apply to everyone where the customer has not set their own (yet).  For this the $GENERATE records are ideal but the choice of what goes in them causes issues.  SORBS looks at these records and if we see things like ’127.0.0.1.static.example.com’ we know the administrators of the network statically assign these networks out to their customers.  If we see ’127.0.0.1.dsl.example.com’ we conclude that the administrators have assigned generic DSL lines to that address space.  Generic DSL being the majority which is of course the home user with a web browser never even considering to run a mail server, but often getting infected with viruses and trojans that send spam, consequently we block it.

Is this SORBS telling you how to run your network?  Not really you can run your network however you want, what SORBS is telling you is if you want to run mail servers from your network we will not accept mail whilst you look like the majority of users (the majority being home users.)  On the other hand, if you want to run your network conforming to best current practices we will be happy to accept mail from you.

To try and simplify things (and cut down the 27,000+ different patterns in use in 2005), staff at SORBS created a draft RFC for defining the ‘tokens’ used in Generic PTR records and whilst in the process of submission it was suggested that it should be submitted as a standard rather than an RFC.  For this to happen the RFC was submitted to the DNS Operations working group, and the original draft allowed to expire.  Unfortunately there are a lot more important issues that are ahead of the proposed standard and it is going to be some years before it is progressed.

Many times we have been accused of telling people to use the wrong SORBS zones, the most pathetic accusation when we were accused of telling everyone to use a particular zone when we showed it in the examples on how to configure SORBS or any other DNSBl.

There are currently many zones available in SORBS, some of which are single zones some of which are aggregate zones (a zone that consists of data from more than one zone.) The main two zones for using SORBS are ‘dnsbl.sorbs.net’ and ‘safe.dnsbl.sorbs.net’ the difference being that one is ‘safer’ to use than the other. ‘safer’ of course is a very objective word and is interpreted in many ways, SORBS uses ‘safer’ in the reduction of false positives (where you might block real mail that you didn’t want to block) the downside being it catches less spam. If you don’t want to evaluate the zones for your own use as we have suggested since the inception of the ‘Using SORBS‘ page and want advice from us, this is how we (the SORBS administrators) use SORBS.

For our mail servers that desire less spam where false positives are not so important, we use ‘dnsbl.sorbs.net’ for default blocking, we also use the ‘cbl.abuseat.org’ as well as ‘level1.bbfh.ext.sorbs.net’. For the mail servers we administer on a corporate level where the directors and CEO’s have specified they’d rather have more spam than the risk of loosing real email we configure ‘safe.dnsbl.sorbs.net’ and ‘cbl.abuseat.org’. In both cases we do not use any pay for services such as Trend Micro’s RBLs and the Spamhaus DNSBls. We use SpamAssassin and ClamAV to scan all the content passing through the servers and use the default DNSBl services included with SpamAssassin. In corporate environments we mark a header for the spam score and have rules on all the internal email servers to automatically put ‘spam email’ in a junk folder. In non corporate environments we configure SpamAssassin into the MTA to reject any message that causes the spam score to exceed the ‘it is spam’ threshold.

For other servers many of the other zones are used, for example a number of chat networks use the zone ‘proxies.dnsbl.sorbs.net’ to block incoming connection requests. Other financial services networks use the ‘web.dnsbl.sorbs.net’ zone to detect and reject trojaned machine connections thereby reducing online financial fraud.

So far we have described how to use SORBS, but just as important is how not to use SORBS.

• Do not use the SORBS DUHL, for blocking connections from your users to your mail servers! This might sound silly, but we have seen it, if you wish to use the SORBS DUHL either whitelist your own users or setup secure connections with SMTP AUTH to bypass the restriction.
• Do NOT use the SORBS DUHL in ‘deep header parsing’ unless it is to increase the likelihood that the message is NOT spam. Again, this might sound very basic and very simple common sense, however the current incarnation of the Barracuda Anti-Spam appliance (April 2010) has been reported as using the SORBS DUHL (amongst other non SORBS dynamic zones) for deep header parsing.
• Do not configure SORBS zones in a way the the blocked person has no idea why they are blocked. This is something that we of SORBS have constant issues with, aside from people messaging us saying they are blocked and the final analysis being that it’s another DNSBl and not SORBS, we also get a lot of people who have no idea what their IP address is. Worse the message they get back states something like, “You message was rejected, reason: blacklisted at SORBS.NET” Now imagine yourself a home user with little understanding of how email works let alone what an IP address is, how are you going to know what you are blocked for?
• Do not use SORBS from third party distribution sites. SORBS provides free access for 99.99% of the world, and as such if you get zones or queries from other systems the chances are the data is already out of date, and worse it may be as much as a few days or weeks old. There are NO Authorised third-party distributors of the SORBS zones.

As we have stated on the ‘Using SORBS’ page the most important thing to do when choosing your anti-spam resources is do your own research. Don’t take our word for it, run your server tagging things that would be blocked by SORBS (and other services) and then check out how good (or bad) we are. Choose then what is good for you, blindly accepting people’s recommendations from the Internet (even us) will result in messages you want being blocked and spam that you want to block getting through.

Sites such as ‘The DNSBl Resource‘ are run by people who work for Email Marketing companies (ESPs) and it is their job to ensure their customers messages get into your inbox. Many such companies claim to be ‘CANSPAM’ compliant, which means they are ‘Opt-Out’ emailers, this is an issue if you are not located in the USA as the law requires they be ‘Opt-In’ only in places such as Europe and Australia. Opt-Out emailing in Australia is categorised as Spam so joining the dots in this section so far we can conclude that it is the purpose of the sites’ owners to ensure the spam they are paid to deliver gets into your inbox.

So what concern is this?

SORBS blocks a lot of these “ESP” companies due to the massive amount of abuse we see (*see footnote) and consequently it is common for them to build a reputation of trying to be fair an honest in their evaluation of which DNSBl based resources to use. The following statement captured from ‘The DNSBl Resource‘ clearly sums up this position and the hidden agenda:

I’ve been working with email senders and email receivers for more than ten years. Time flies when you’re having fun, helping the good guys block unwanted mail and pressuring the bad guys to reduce false positive blocking.

Which means he (the site owner: Al Iverson) works with anyone that doesn’t block his services to try and force those who do block his services to allow his companies email through.

SORBS’ view is clear in this matter, research yourself as there are many people out there who want you to work to their agenda. SORBS’ agenda is simple, we want people to stop sending spam. We don’t care how much money Professional/Legal spammers/Email senders loose, just stop filling our mailboxes and those of others with junk. We don’t want it.

* One of the ESPs who say they are ‘anti-spam’ and ‘opt-in’ only managed 30,000 individual spams to one of our servers in a 24 hour period, this was exceptional the normal load being around 1,000 per day but clearly shows the issue.

The SORBS Spam Database is not a pleasant place to be listed, but it serves its purpose and highlights issues where a host may have a compromised web script.

The other thing the Spam Database includes is networks where spammers reside (or have servers) that the network owner or ISP allows and has refused to terminate their services… More on that issue and what you can do about it can be found later in this article.

So how does the SORBS Spam Database work… well quite simply it lists the IP addresses we have received spam from directly. This could mean a compromised host, it could mean the ISPs mail server for it’s users or it could mean a shared host where there is a script (PHP and Perl are the most common, but ASP and .NET have also been seen) that can be abused to allow mail to be sent on behalf of spammers.

The SORBS Spam database is split into 4 different datasets (Zones or Databases if you like) each described in this simple list:

      new.spam.dnsbl.sorbs.net - IPs sending spam in the last 48 hours.
   recent.spam.dnsbl.sorbs.net - IPs sending spam in the last 28 days.
      old.spam.dnsbl.sorbs.net - IPs that have sent spam within the last year (365 days.)
	  spam.dnsbl.sorbs.net - IPs that have sent spam in the past (no time limit.)
   escalations.dnsbl.sorbs.net - Networks that have sent spam or are sending spam or are
                                 where other spam services are hosted.

 
You might ask, “What does ‘where other spam services are hosted’ mean?” Well simply it means the spammer might be hosting their website(s) there, they might be hosting other things like their DNS servers there, all of which help the spammer to continue to spam. You might ask, “What’s the point if they are not sending mail from there?” Well, we (SORBS) want the ISPs and providers to get rid of the spammers from their networks and refuse to provide hosting for them, after all what’s the point of sending the spam if they can’t sell anything when the minority of people reply to it? Occasionally we will list larger chunks of a network based solely on spam, this is where we have seen a continuous stream of spam from IPs and occasionally they move to “get around” the listings. This is also an ISP either knowingly or unknowingly supporting the spammer on their network.

A spam escalation netblock is not a pleasant place to be, but we (SORBS) use it as a last resort to get the attention of the provider and try to give gentle pressure to the provider to get rid of the spammer. If you find yourself the subject of a listing what follows are are some suggestions as to what you can do:

• Demand your provider terminate the spammers or the support services. This is an easy way to find out what business your provider is in, but beware, the common responses are:
  • “SORBS will not delist, we’ve tried” – This means they talked to us and we told them they need to terminate the spammers and they refused.
  • “SORBS requires a fine and we won’t pay it” – This means they know and accept that spam was sent, but they are unwilling to take responsibility for that spam even though it was their servers that sent it.
  • “We can’t contact anyone at SORBS” – well as you already know by talking to us this is not true, anyone can log a ticket though the SORBS Support System
• Move to another provider.
• Use SMTP Services of another provider and the “smarthost” functionality of your mail server software to send email from another relay that you are authorised to use.

Of course in the first point, these are the most common responses and in each case are an excuse to continue getting money from the spammers. What most people don’t know is spammers will often provide, “Incentives” (aka bribes) to the sales man that has sold them the services to ignore the spam complaints. The net result is the ISP/provider can be getting as much as 5 times the normal rate for the hosting so they don’t want to terminate the spammer. This means the spammer’s business is more important to them than yours. They are also banking on the fact that once hosted, you might complain about blocking, but in reality you don’t want to move because you have a good deal or you perceive it’s just too expensive for you to move. SORBS accepts this as a problem, but stands firm in that we need to keep the pressure up on the providers, our customers support us in this stance and unless you’re happy to get more spam you should to.
So what happens if you have a single IP on the list, well this usually means that your server has sent spam, and getting off the list varies depending on the type and frequency of the spam, whether you are the new owner of the netblock or other circumstances. In each case we try to evaluate your situation and we will act accordingly. The Spam Database FAQ is quite firm in it’s stance and our policy but in reality this is to make our lives easier. We do evaluate listings individually, we will delist people ‘free of charge’ (once) and we will delist old entries. If however we think that the spam issue is just starting from your IP, or we believe you haven’t actually done anything to ensure it happens again, we will fall back to the policy and require a fine to be paid before we delist you. If you believe this to be extortion or blackmail you should visit the article that discusses the issue.

Something we haven’t told people, but is often asked is, “How is the data compiled?” This article will touch on the subject,and will give some detail, which if used right you can ensure you are never listed by SORBS, or the listings are very rare.

The vase majority of entries (well over 2 million as of April 2010) are listed because we received spam to one of our spamtrap servers. The host was identifying spam because it delivered spam with a known Spam URI. “What?” Well simply we receive around 10 million emails a day to disused domains, these might be recently expired, they might be domains that have been expired for years, and we decode the body of the messages, process any javascript in a sandbox, and then check any URIs we find against the SURBl and URIBl as well as our own (currently) internal list. If we find a match we wrap up the spam, checksum it, and send it to our ‘spam processing servers’. These servers check the host sending the message is authorised to send spam reporting messages, it checks the checksum of the message to ensure it has not been tampered with, and then it unwraps the message and inserts it directly into the Spam Database together with the checksum, we will (soon) write these messages directly to DVDR (write once). All of these measures ensure that the spam recorded actually came from one of our servers and its origin has been recorded securely for law enforcement research and to prevent forgeries causing listings.

Other methods of determining spam, are more simple, we use people. We have a system which interfaces to the same wrap up method we use for the spamtrap servers, however the SORBS admins from around the world and many professions have the task of sorting mail from spam. What happens is the spam we get in our Inboxes is moved (by hand) to a folder called, “Spam4SORBS” or “Spam” or even “Junk Mail” and every 5 minutes a “robot” logs into the server as the user, or the administrator checks those folders and grabs all the messages, wrapping them and sending them to the spam processing servers.

Lastly there are web forms available, such as the Spam Submission Beta test and in our admin interface where we can either cut/paste spam or we can create network listings of 1 IP through to 65536 IPs or more. All the listings that are not for a single IP are checked by the other SORBS administrators for detail and mistakes, it is also to prevent a repeat of the past where someone who was working for another anti-spam service was able to create 3 very large listings which had an immediate and detrimental effect on the SORBS service as well as upsetting millions of users of email who were blocked. Note: we also do background checks on all our new staff and have them sign contracts which should prevent a re-occurrence.

So how do you get out? Well simply, you have to follow the SORBS Spam Database FAQ or convince one of our staff that you and your IP are not going to spam again. In reality we know this you can’t guarantee, but you need to take measures to minimise the issue. It’s no go saying, “We found a virus on one of our machines an removed it” because that doesn’t help stop it re-occurring, basic network hygiene dictates that you should have a up to date anti-virus software (with current definitions) running on ALL your hosts. You should be patching your machines regularly and using tools from vendors to ensure that the patches are applied. If you are not doing this already, then that is the likely cause of the problem.

Consider the basic security implications of getting a virus that sends spam… If you are infected and the host is sending spam, it could also have key-logged everything on that server, and sent it to the virus creator, it could have sent most of your corporate secrets. It could also have sent compromising photos which could be used to blackmail you later… “That’s just scare tactics” you can shout, but every one of those scenarios have already been seen in the real world. People have had their banking details stolen and thousands of dollars sent to money laundering accounts in Russia. Business mean that are also cross-dressers, and VIPs that are cheating on their partners have been blackmailed into sending thousands of dollars to people around the world. Ideas and works that are about to be patented have been stolen and sold to the highest bidder.. Spam is only what keeps a regular income, a sideline you might say, the real money is in the scams, the emptying of bank accounts and blackmail, all of which is run by organised crime. The people behind the spam and viruses are people who have a lot of money to pay for developers to come up with new ways to get into your systems and steal from you. No longer is spam just about email, its about money, real money, and lots of it.

The SORBS database is a tool you can use to help identify where you went wrong as it will flag problems within minutes. This might be too late for your personal data, but it is better to be forewarned as you can be forearmed. The Spam database is there to stop the flow of spam (and sometimes viruses) into the networks of SORBS users giving them another line of defense. This might be an inconvenience for you in the short term, but consider the implications to them and you if they are infected by something you have sent.

I wonder how long it will be before corporate America has some big scandal where the result is some one who hasn’t taken measures to protect their network is sued for negligence when that network successfully attacks someone resulting in a lose of corporate secrets worth millions…?

This article is for all to read, but you should also consider that only a small part of what constitutes of the SORBS Database is the SORBS Spam Database. If you are not familiar with this distinction (and most people are not) please read the article on the SORBS Spam Database that defines what the SORBS Spam Database actually is.

Many people accuse SORBS of extortion or blackmail, in nearly all cases the people promoting it as such are people who are listed in the spam database for sending spam, most are the spammers themselves. Spammers go by multiple guises and will create multiple blogs, post (news group, forums and blog comments) accusing SORBS of extortion or blackmail to try and discredit SORBS so that fewer people use SORBS which means their spam can get through. There are the occasional people that do the same who are not spammer, but mis-guided by the myriad of posts by the spammers.

Here are some truths and the reality of the situation.

SORBS blocks nothing, the administrators (including those of the SORBS mail servers) use SORBS to decided whether to allow email through or reject it. This is the fundamental and legal point as to why SORBS is not blackmail or extortion. Of course there are those who disagree, but consider that SORBS has operated in multiple jurisdictions of the world, all with laws against extortion and blackmail, all where people have complained to law enforcement and consulted lawyers about private prosecutions. As of April 2010, SORBS has been operating the same policy for seven years and has never even had a court case started let alone lost for extortion or blackmail. We have never let anyone out because of pressure, indeed when we listed one of the mail servers for the Queensland (Australia) Police force I (Michelle Sullivan) received phone call threats from a senior sergeant about the listing. I explained the policy and explained why it was listed, and and what they needed to do to get the IP delisted, he argued for a few hours. A few days later the IP was delisted when they complied with the policy requirements. A similar issue happened with a number of departments of Australian Federal government (whom I later worked for) it caused a policy change for their email systems which now results in a significant reduction in spam from their hosts.

So why is it not extortion you ask? Well aside from the legal point I mentioned above, the definition of Extortion is (according to wikipedia) as follows:

Extortion, outwresting, or/and exaction is a criminal offense which occurs when a person unlawfully obtains either money, property or services from a person(s), entity, or institution, through coercion. Refraining from doing harm is sometimes euphemistically called protection. Extortion is commonly practiced by organized crime groups. The actual obtainment of money or property is not required to commit the offense. Making a threat of violence which refers to a requirement of a payment of money or property to halt future violence is sufficient to commit the offense. Exaction refers not only to extortion or the unlawful demanding and obtaining of something through force,^[1] but additionally, in its formal definition, means the infliction of something such as pain and suffering or making somebody endure something unpleasant.^[2]

In the United States, extortion may also be committed as a federal crime across a computer system, phone, by mail or in using any instrument of “interstate commerce.” Extortion requires that the individual sent the message “willingly” and “knowingly” as elements of the crime. The message only has to be sent (but does not have to reach the intended recipient) to commit the crime of extortion.

Extortion is distinguished from robbery. In “strong arm” robbery, the offender takes goods from the victim with use of immediate force. In “robbery” goods are taken or an attempt is made to take the goods against the will of another—with or without force. A bank robbery or extortion of a bank can be committed by a letter handed by the criminal to the teller. In extortion, the victim is threatened to hand over goods, or else damage to their reputation or other harm or violence against them may occur. Under federal law extortion can be committed with or without the use of force and with or without the use of a weapon. A key difference is that extortion always involves a written or verbal threat whereas robbery can occur without any verbal or written threat (refer to U.S.C. 875 and U.S.C. 876).

The term extortion is often used metaphorically to refer to usury or to price-gouging, though neither is legally considered extortion. It is also often used loosely to refer to everyday situations where one person feels indebted against their will, to another, in order to receive an essential service or avoid legal consequences. For example, certain lawsuits, fees for services such as banking, automobile insurance, gasoline prices, and even taxation, have all been labeled “legalized extortion” by people with various social or political beliefs.

Neither extortion nor blackmail require a threat of a criminal act, such as violence, merely a threat used to elicit actions, money, or property from the object of the extortion. Such threats include the filing of reports (true or not) of criminal behavior to the police, revelation of damaging facts (such as pictures of the object of the extortion in a compromising position), etc.

Now why SORBS often gets confused with extortion is because of the fourth paragraph, people feel indebted against their will, because they cannot send email to SORBS’ users without paying the ‘fine’ to get off the list. Problem is that is true (for the Spam Database), they cannot without paying the fine, and whilst that might leave a bad feeling for them, their IP address committed an act (in most case) which cost SORBS money. Their server accessed our servers either deliberately, accidentally or maliciously and used it’s bandwidth and resources against our acceptable use policy, against our wishes and without our permission. We would be well within our rights as service providers to charge someone real money for that usage (authorised or not) however we choose instead to make a requirement for the fine payers to donate to charity or a good cause. Our legal advice has told us this is fine to do, they have also indicated we could legally charge for that unauthorised access, but they added that whilst we could do it legally it would likely enable people to take us to court more easily and whilst there is legal basis for us to win every time, we would have to defend that, and defending law issues take time and cost money. We decided long ago that we should have the money donated to charity as they are plenty of worthy causes out there that are in dire need of financial aid

• Information copied from Wikipedia courtesy of the creative commons and share-alike license.

It seems this topic keeps coming up with a regularity that is surprising if annoying at times. People seem to confuse the SORBS databases and get angry about the fine applying to the DUHL and Proxy entries etc. This of course is pointless, SORBS only charges for removal from the SORBS spam database and the other databases have their own delisting policies.

Herein lies the problem, many people think the SORBS Spam DB means any listing in the SORBS database, of course it is completely wrong. For those with a technical background SORBS v1.0 has several tables in a single database, each table holds data about IP addresses and networks, these are exported to the RSYNC and DNS servers every minute into different ‘zones’.

A detailed explanation of what each zone is and what it contains can be found on the Using SORBS page, what follows is a list of the individual zones and what they are called:

http.dnsbl.sorbs.net - This is the SORBS HTTP Proxy Database
socks.dnsbl.sorbs.net - This is the SORBS SOCKS Proxy Database
misc.dnsbl.sorbs.net - This is the SORBS Miscellaneous Proxy Database
smtp.dnsbl.sorbs.net - This is the SORBS Open-Relay Database
new.spam.dnsbl.sorbs.net - This is the SORBS Spam Database
recent.spam.dnsbl.sorbs.net - This is the SORBS Spam Database
old.spam.dnsbl.sorbs.net - This is the SORBS Spam Database
escalations.dnsbl.sorbs.net - This is the SORBS Spam Database
web.dnsbl.sorbs.net - This is the SORBS Web and Vulnerability Database
block.dnsbl.sorbs.net - This is the SORBS Admin Block Database
zombie.dnsbl.sorbs.net - This is the SORBS Zombie Network Database

From this quick look you can see that the Spam Database is only a small part of what SORBS offers.

What does this mean to you? Well quite simply if you are blocked by SORBS unless your database lookup says ‘Spam Database’ then you have nothing to pay, there is no fine.

Now, what are all the databases?

The HTTP, SOCKS and MISC databases are all proxy servers of one sort or another, and getting delisted is a simple matter of obtaining a key, and sending in a specifically formatted message to our test address. This will cause servers around the world to issue random tests on the server immediately and if it appears not to have a proxy anymore, it will delist the IP address. Over the following few weeks other servers will perform the same tests at random to help ensure that the proxy server was secured and not just ‘turned off’.

The Open-Relay database is similar to the proxy database in that the delisting function works in the same way, however rather than testing for an open proxy it tests to see if the server is a mail server that will relay messages for anyone.

The Web and Vulnerability database is a little secretive about how it works, but here’s the gist… If your host sends messages into our spamtraps the connection information is checked, this is things like the TCP Flags, the Hostname, the IP address, the SMTP commands it uses and which order, and the number of times it attempts to connect in a certain period of time. If certain conditions are met, the host is listed as a ‘Possibly Trojaned Host’. Additionally if the host attempts to send viruses to the spamtrap servers it could also cause a listing, though these are more stringently checked as we do not wish to list ISPs mail servers for delivering virus payloads even though they should be virus scanning ALL mail.

The Zombie Database is not a database of hosts that are zombie machines, but are of networks that have been hijacked. Hijacking occurs when a network becomes disused or the owner fall into receivership and a spammer takes over their network by fraudulent means. A lot of research goes into entries in this database, and removal is more research which a listee can aid by proving their legal ownership of the domain concerned.

The admin block database is where the administrator of the network has requested that they never be contacted by anyone at SORBS or any of the test machines, and to prevent SORBS servers from being triggered into sending messages or contacting their networks a general block is placed on every service. Removal can happen at any time when the registered network owner requests delisting.

The Spam Database, well this one is where we put IPs sending spam, and any netblocks that are actively supporting spammers. This means that occasionally people are blocked because of other people within their network. More information on this issue can be found in our ‘Spam Database Listings‘ article.

I hope this gives a little insight into how the SORBS Databases are defined and you will now understand that a listing in SORBS doesn’t mean you’re “listed on the spam database” but can mean a variety of other things.