Spam Filters Explained


What do they do? How do they work? Which one is right for me?By Alan Hearnshaw

Spam is a very real problem that many people have to deal with on a daily basis. For those that have decided to do something about it and start to investigate the options available in spam filtering, this article provides a brief introduction to your options and the types of spam filters available.

Despite the bewildering array of spam filters available today, all claiming to the best one "of its kind" there are really just five filtering methodologies in general use today and all products rely on one, or a combination of these:

Content-Based Filters

"In the beginning, there were content-based filters."

These filters scan the contents of the and look for tell-tale signs that the message is spam. In the early days of spamming it was quite simple to look out for "Kill Words" such as "Lose Weight" and mark a message as spam if it was found.

Very soon though, spammers got wise to this and started resorting to all kinds of tricks to get their message past the filters. The days of "obfuscation" had begun. We started getting messages containing the phrase "L0se Welght" (Notice the zero for "o" and "l" for "i") and even more bizarre - and sometimes quite ingenious - variations.

This rendered basic content-based filters somewhat ineffective, although there are one or two on the market now that are clever enough to "see through" theses attempts and still provide good results.

Bayesian Based Filters

"The Reverend Bayes comes to the rescue"

Born in London 1702, the son of a minister, Thomas Bayes developed a formula which allowed him to determine the probability of an event occurring based on the probabilities of two or more independent evidentiary events.

Bayesian filters "learn" from studying known good and bad messages. Each message is split into single "word bytes", or tokens and these tokens are placed into a database along with how often they are found in each kind of message.

When a new message arrives to be tested by the filter, the new message is also split into tokens and each token is looked up in the database. Extrapolating results from the database and applying a form of the good reverend's formula, know as the a "Naive Bayesian" formula, the message is given a "spamicity" rating and can be dealt with accordingly.

Bayesian filters typically are capable of achieving very good accuracy rates (>97% is not uncommon), and require very little on-going maintenance.

Whitelist/Blacklist Filters

"Who goes there, friend or foe?"

This very basic form of filtering is seldom used on its own nowadays, but can be useful as part of a larger filtering strategy.

A "whitelist" is nothing more than a list of e-mail addresses from which you wish to accept communications. A whitelist filter would only accept messages from these people and all others would be rejected

A "blacklist", conversely, is a list of e-mail addresses - and sometimes IP Addresses (computer identification addresses) - from which communications will not be accepted.

While this may seem like a good idea from the outset, a whitelist methodology is too restrictive for most people and, as virtually all spam e-mails carry a forged "from" address, there is little point in collecting this address to ban it in future as it is very unlikely to be the same next time.

There are bodies on the internet that maintain a list of known "bad" sources of e-mail. Many filters today have the ability to query these servers to see if the message they are looking at comes from a source identified by this Internet-based blacklist, or RBL. While being quite effective, they do tend to suffer from "false positives" where good messages are incorrectly identified as spam. This happens often with newsletters.

Challenge/Response Filters

"Open sesame!"

Challenge/Response filters are characterised by their ability to automatically send a response to a previously unknown sender asking them to take some further action before their message will be delivered. This is often referred to as a "Turing Test" - named after a test devised by British mathematician Alan Turing to determine if machines could "think".

Recent years have seen the appearance of some internet services which automatically perform this Challenge/Response function for the user and require the sender of an e-mail to visit their web site to facilitate the receipt of their message.

Critics of this system claim it to be too drastic a measure and that it sends a message that "my time is more important than yours" to the people trying to communicate with you.

For some low traffic e-mail users though, this system alone may be a perfectly acceptable method of completely eliminating spam from their inbox - one step above the "Whitelist" system outlined above.

Community Filters

"A united front"

These types of filters work on the principal of "communal knowledge" of spam. When a user receives a spam message, they simply mark it as such in their filter. This information is sent to a central server where a "fingerprint" of the message is stored.

After enough people have "voted" this message to be spam, then it is stopped from reaching all the other people in the community.

This type of filtering can prove to be quite effective, although it stands to reason that it can never be 100% effective as a few people have to receive the spam for it to be "flagged" in the first place. Just like its similar cousin the Internet black list (RBL), this system also can suffer from "false positives", or messages incorrectly identified as spam.

Hopefully you are now armed with a little more information to be able to make an informed decision on the best spam filter for you.

For further information, consider reading the reviews and articles found at http://www.whichspamfilter.com

Alan Hearnshaw is a computer programmer and the owner of http://www.whichspamfilter.com, a web site which conducts weekly in-depth reviews of current spam filters, provides help and guidance in the fight against spam and provides a useful community forum.alan@whichspamfilter.com





Latest News:




Warning: MagpieRSS: Failed to parse RSS file. (syntax error at line 1, column 0) in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/magpierss/rss_fetch.inc on line 238
Site:

Warning: Invalid argument supplied for foreach() in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/inc/ads-body.inc on line 34


Site: news - Google News
US urges calm as Kirkuk crisis escalates - BBC News

BBC News

US urges calm as Kirkuk crisis escalates
BBC News
The US has called for "calm" after Iraqi government forces seized the northern city of Kirkuk and key installations from Kurdish control. State department spokeswoman Heather Nauert urged all parties to "avoid further clashes". Iraqi soldiers moved ...
Baghdad: Iraqi forces in full control of KirkukAljazeera.com

all 1,711 news articles »
North Korea: 'Nuclear war may break out any moment' - Fox News

Fox News

North Korea: 'Nuclear war may break out any moment'
Fox News
Rep. Peter King (R-N.Y.) weighs in on President Trump's tough stance on North Korea, and why the could be the first president to successfully bargain with the Hermit Kingdom. North Korea told other countries on Monday to avoid participating in military ...
North Korea: Nuclear war may break out at 'any moment'Aljazeera.com
UK TV drama about North Korea hit by cyber-attackBBC News
North Korea's economy may not survive another year, defector saysCNBC

all 764 news articles »
Bowe Bergdahl pleads guilty in desertion, misbehavior case - ABC News

Fox News

Bowe Bergdahl pleads guilty in desertion, misbehavior case
ABC News
In an interview recorded last year and obtained by ABC News, Bergdahl said that because of Trump's statements, he did not believe he would be able to receive a fair trial. "We may as well go back to kangaroo courts and lynch mobs that got what they ...
Bowe Bergdahl pleads guilty in desertion caseFox News
Army deserter Bowe Bergdahl doubts he could get a fair trial after Trump commentsABC News
Bowe Bergdahl pleads guilty to desertion, faces up to life in prisonCNN
New York Times
all 211 news articles »
The Fox News president - Washington Post

Washington Post

The Fox News president
Washington Post
Since his election as president, Donald Trump has tweeted to or about Fox News nearly 130 times. The most recent example was early Monday morning, when Trump shared a comment from economist Art Laffer that had just been made on a Fox network.

and more »
Restaurant chain 'cut sugary drink sales' with price rise - BBC News

BBC News

Restaurant chain 'cut sugary drink sales' with price rise
BBC News
Prof Susan Jebb, professor of diet and population health at the University of Oxford, said the findings were "encouraging news for public health". But she said there was a disappointing lack of data on alcohol sales, which could have increased over the ...

and more »
Las Vegas guard Jesus Campos vanished after visiting urgent-care clinic, union leader says - Fox News

Fox News

Las Vegas guard Jesus Campos vanished after visiting urgent-care clinic, union leader says
Fox News
A spokesperson at the UMC Quick Care, which has eight locations throughout the Las Vegas area, told Fox News on Monday that they had "heard nothing" about Campos visiting them. Multiple requests from Fox News for SPFPA to comment on the matter ...

and more »
Bombardier to partner Airbus on C-Series jets - BBC News

BBC News

Bombardier to partner Airbus on C-Series jets
BBC News
Bombardier's Northern Ireland's director Michael Ryan said the deal was "great news" for the Belfast operation. About 1,000 staff work on the C-Series at a purpose-built factory in Belfast, mostly making the plane's wings. Airbus and Bombardier's chief ...

and more »
Weinstein scandal puts several movies on hold - Fox News

Fox News

Weinstein scandal puts several movies on hold
Fox News
Harvey Weinstein's downfall may also down his slate of upcoming movies ? including his solo live-action directorial debut. Weinstein announced in August that he would direct ?Mila 18,? an adaptation of Leon Uris' novel about the Warsaw Ghetto Uprising.

and more »
Trump suggests Obama didn't call families of fallen soldiers as president - CBS News

CBS News

Trump suggests Obama didn't call families of fallen soldiers as president
CBS News
A former White House official also told CBS News, "President Trump's claim is wrong. President Obama engaged families of the fallen and wounded warriors throughout his presidency through calls, letters, visits to Section 60 at Arlington, visits to ...
Trump Falsely Claims Obama Didn't Contact Families of Fallen TroopsNew York Times

all 202 news articles »
McCain condemns isolationist politics, calls it 'unpatriotic' - Fox News

Fox News

McCain condemns isolationist politics, calls it 'unpatriotic'
Fox News
Sen. John McCain, R-Ariz., right, accompanied by Chair of the National Constitution Center's Board of Trustees, former Vice President Joe Biden, waves as he takes the stage before receiving the Liberty Medal in Philadelphia, Monday, Oct. 16, 2017. (AP).

and more »

MORE RESOURCES:

Warning: MagpieRSS: Failed to parse RSS file. (syntax error at line 1, column 0) in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/magpierss/rss_fetch.inc on line 238
Site:

Warning: Invalid argument supplied for foreach() in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/inc/rss.php on line 9
Site: spam blocker - Google News
How to Block SMS Spam on iOS 11 - NDTV

NDTV

How to Block SMS Spam on iOS 11
NDTV
SMS spam is something that plagues pretty much everyone who has an Indian SIM card, and even those who live abroad. Be it frauds, asking you to buy shares in companies to double your wealth, or just your neighbourhood store sending promotional ...

A Spam Story - How One Law Firm's Spam Blocker Proved Costly to Them and to Their Client - JD Supra (press release)

JD Supra (press release)

A Spam Story - How One Law Firm's Spam Blocker Proved Costly to Them and to Their Client
JD Supra (press release)
The first, is that they had to choose how to configure their spam blocker. The easiest and most cost-efficient way was to rely fully upon the Microsoft Exchange server which rejected and permanently deleted messages that appeared to be spam. However ...

VIPRE Advanced Security review - TechRadar

TechRadar

VIPRE Advanced Security review
TechRadar
... now offers a range of products for home and business users. VIPRE Advanced Security is the company's consumer-oriented security suite, a blend of dual-engine antivirus (Bitdefender and its own), firewall, spam blocker, web filter and basic privacy ...

Google got duped by a fake ad blocker, and 37000 people accidentally downloaded adware - Mashable

Mashable

Google got duped by a fake ad blocker, and 37000 people accidentally downloaded adware
Mashable
Some 37,000 people downloaded a spam version of the "AdBlock Plus" extension from Google after a fraudulent ad blocking extension snuck through Google Chrome's verification process and appeared in the Chrome Web Store yesterday. It's easy to see ...

Google Chrome users tricked into downloading fake ad blocker - The Independent

The Independent

Google Chrome users tricked into downloading fake ad blocker
The Independent
... Plus program, which is developed by adblockplus.org and has over 10 million users. ?This is not the legitimate version of Adblock Plus,? he wrote. "This is a spam account. The legit version of Adblock Plus is the one at the top of the Chrome ...
SwiftOnSecurity on Twitter: "Google allows 37,000 Chrome users to be tricked with a fake extension by fraudulent ...Twitter
37000 Chrome users downloaded a fake Adblock Plus extensionEngadget
Adblock Plus and (a little) more: 37,000 fooled after ABP imposter gets in Web Store; how to make sure you've got ...Adblock Plus
Chrome - Google
all 66 news articles »
SMS Shield ? SMS Spam Filtering Comes to iOS 11 - Beautiful Pixels (blog)

Beautiful Pixels (blog)

SMS Shield ? SMS Spam Filtering Comes to iOS 11
Beautiful Pixels (blog)
With a few simple configuration tweaks, it has managed to handle most of the spam I've been getting on my iPhone. You can customize SMS Shield to filter out unwanted messages by setting up your own rules too. It offers a Custom Block List feature that ...

CallApp Introduces Call Recorder with a First Ever Fully Free Caller ID & Spam Blocker - PR Newswire (press release)

PR Newswire (press release)

CallApp Introduces Call Recorder with a First Ever Fully Free Caller ID & Spam Blocker
PR Newswire (press release)
NEW YORK, Sept. 19, 2017 /PRNewswire/ -- CallApp, a worldwide leader in Caller ID services, makes history today by adding a Call Recorder to its Contacts, Dialer & Caller ID App. CallApp will now offer the full package of Caller ID and Call Recording ...

and more »
How iOS 11 SMS messaging filters works - Macworld - Macworld

How iOS 11 SMS messaging filters works - Macworld
Macworld
iOS 11 now allows you to send incoming SMS messages through third-party filters, just like spam processing in email programs. The good ones are filtered into ...

and more »
Google Joins Mobile Payments Race in Cash-Reliant India - Bloomberg

Financial Express

Google Joins Mobile Payments Race in Cash-Reliant India
Bloomberg
Spam-blocker app Truecaller already offers in-app payments as does homegrown messaging service Hike, backed by Tencent Holdings Ltd. and Foxconn Technology Group. Several Indian banks have their own digital payments apps, which earn revenue by ...
Tez | Tez For BusinessTez - Google
Introducing Tez. Created by Google, made for India first. - YouTubeYouTube
Tez ? A new payments app by Google - Android Apps on Google PlayGoogle Play

all 405 news articles »
Google Chrome Will Block Tab-Under Behavior - BleepingComputer

BleepingComputer

Google Chrome Will Block Tab-Under Behavior
BleepingComputer
Telemetry gathered by Google showed that a big percentage of the original pages (now showing ads or another website) are never engaged with again, meaning they're obviously abused to spam users. Google engineers have grown tired of this behavior ...

Home | Site Map | Resource Links