Spam Filters Explained


What do they do? How do they work? Which one is right for me?By Alan Hearnshaw

Spam is a very real problem that many people have to deal with on a daily basis. For those that have decided to do something about it and start to investigate the options available in spam filtering, this article provides a brief introduction to your options and the types of spam filters available.

Despite the bewildering array of spam filters available today, all claiming to the best one "of its kind" there are really just five filtering methodologies in general use today and all products rely on one, or a combination of these:

Content-Based Filters

"In the beginning, there were content-based filters."

These filters scan the contents of the and look for tell-tale signs that the message is spam. In the early days of spamming it was quite simple to look out for "Kill Words" such as "Lose Weight" and mark a message as spam if it was found.

Very soon though, spammers got wise to this and started resorting to all kinds of tricks to get their message past the filters. The days of "obfuscation" had begun. We started getting messages containing the phrase "L0se Welght" (Notice the zero for "o" and "l" for "i") and even more bizarre - and sometimes quite ingenious - variations.

This rendered basic content-based filters somewhat ineffective, although there are one or two on the market now that are clever enough to "see through" theses attempts and still provide good results.

Bayesian Based Filters

"The Reverend Bayes comes to the rescue"

Born in London 1702, the son of a minister, Thomas Bayes developed a formula which allowed him to determine the probability of an event occurring based on the probabilities of two or more independent evidentiary events.

Bayesian filters "learn" from studying known good and bad messages. Each message is split into single "word bytes", or tokens and these tokens are placed into a database along with how often they are found in each kind of message.

When a new message arrives to be tested by the filter, the new message is also split into tokens and each token is looked up in the database. Extrapolating results from the database and applying a form of the good reverend's formula, know as the a "Naive Bayesian" formula, the message is given a "spamicity" rating and can be dealt with accordingly.

Bayesian filters typically are capable of achieving very good accuracy rates (>97% is not uncommon), and require very little on-going maintenance.

Whitelist/Blacklist Filters

"Who goes there, friend or foe?"

This very basic form of filtering is seldom used on its own nowadays, but can be useful as part of a larger filtering strategy.

A "whitelist" is nothing more than a list of e-mail addresses from which you wish to accept communications. A whitelist filter would only accept messages from these people and all others would be rejected

A "blacklist", conversely, is a list of e-mail addresses - and sometimes IP Addresses (computer identification addresses) - from which communications will not be accepted.

While this may seem like a good idea from the outset, a whitelist methodology is too restrictive for most people and, as virtually all spam e-mails carry a forged "from" address, there is little point in collecting this address to ban it in future as it is very unlikely to be the same next time.

There are bodies on the internet that maintain a list of known "bad" sources of e-mail. Many filters today have the ability to query these servers to see if the message they are looking at comes from a source identified by this Internet-based blacklist, or RBL. While being quite effective, they do tend to suffer from "false positives" where good messages are incorrectly identified as spam. This happens often with newsletters.

Challenge/Response Filters

"Open sesame!"

Challenge/Response filters are characterised by their ability to automatically send a response to a previously unknown sender asking them to take some further action before their message will be delivered. This is often referred to as a "Turing Test" - named after a test devised by British mathematician Alan Turing to determine if machines could "think".

Recent years have seen the appearance of some internet services which automatically perform this Challenge/Response function for the user and require the sender of an e-mail to visit their web site to facilitate the receipt of their message.

Critics of this system claim it to be too drastic a measure and that it sends a message that "my time is more important than yours" to the people trying to communicate with you.

For some low traffic e-mail users though, this system alone may be a perfectly acceptable method of completely eliminating spam from their inbox - one step above the "Whitelist" system outlined above.

Community Filters

"A united front"

These types of filters work on the principal of "communal knowledge" of spam. When a user receives a spam message, they simply mark it as such in their filter. This information is sent to a central server where a "fingerprint" of the message is stored.

After enough people have "voted" this message to be spam, then it is stopped from reaching all the other people in the community.

This type of filtering can prove to be quite effective, although it stands to reason that it can never be 100% effective as a few people have to receive the spam for it to be "flagged" in the first place. Just like its similar cousin the Internet black list (RBL), this system also can suffer from "false positives", or messages incorrectly identified as spam.

Hopefully you are now armed with a little more information to be able to make an informed decision on the best spam filter for you.

For further information, consider reading the reviews and articles found at http://www.whichspamfilter.com

Alan Hearnshaw is a computer programmer and the owner of http://www.whichspamfilter.com, a web site which conducts weekly in-depth reviews of current spam filters, provides help and guidance in the fight against spam and provides a useful community forum.alan@whichspamfilter.com





Latest News:




Warning: MagpieRSS: Failed to parse RSS file. (syntax error at line 1, column 0) in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/magpierss/rss_fetch.inc on line 238
Site:

Warning: Invalid argument supplied for foreach() in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/inc/ads-body.inc on line 34


Site: news - Google News
This RSS feed URL is deprecated
This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news Trump lawyer accuses Mueller of improperly obtaining transition emails - NBCNews.com

NBCNews.com

Trump lawyer accuses Mueller of improperly obtaining transition emails
NBCNews.com
GSA deputy counsel Lenny Loewentritt disputed the claims in the letter, telling Buzzfeed News that the Trump transition team were told any material ?would not be held back in any law enforcement? eventuality. ?Therefore no expectation of privacy can be ...
Firing special counsel Mueller 'would be a mistake': Leading GOP senatorABC News
Gregg Jarrett: Mueller's allegedly lawless acts have corrupted his probe and demand his removalFox News
How Fox News and President Trump create an anti-Mueller ...CNNMoney
CBS News -Aljazeera.com -Axios -KQED
all 215 news articles »
Sell the News? Bitcoin Price to Test $20k Ahead of CME Launch - Coindesk

Coindesk

Sell the News? Bitcoin Price to Test $20k Ahead of CME Launch
Coindesk
The upside move in bitcoin has gathered traction this weekend as investors brace up for BTC futures launch by the CME. As per CoinDesk's Bitcoin Price Index (BPI), the world's largest cryptocurrency by market capitalization rose to a new all-time high ...

and more »
CIA 'helped stop Russia terror attack' - BBC News

CIA 'helped stop Russia terror attack'
BBC News
Information provided by the CIA helped Russian security services foil an attack on St Petersburg's Kazan cathedral, the Kremlin says. The attack was allegedly planned to take place on Saturday, officials say. In a phone call, President Vladimir Putin ...

and more »
Democrats hold biggest lead in congressional preference since 2008 - NBCNews.com

NBCNews.com

Democrats hold biggest lead in congressional preference since 2008
NBCNews.com
WASHINGTON ? Fresh off their victory in Alabama's special Senate election, Democrats now enjoy their largest advantage in congressional preference in nine years, according to the latest NBC News/Wall Street Journal poll, signaling a dangerous ...

and more »
Deadly attack on Methodist church in Pakistan - BBC News

Aljazeera.com

Deadly attack on Methodist church in Pakistan
BBC News
A suicide bomb and gun attack on a Methodist church in Pakistan has killed at least eight people, officials say. Militants stormed the church, which was packed with worshippers at the time, in the city of Quetta, some 65km (40 miles) from the Afghan ...
Suicide bombers attack Pakistan church, killing at least 9, officials sayFox News
Pakistan: Quetta church hit in suicide attackAljazeera.com

all 192 news articles »
Campaigner predicts North Korea 'peace offensive' despite US standoff - NBCNews.com

NBCNews.com

Campaigner predicts North Korea 'peace offensive' despite US standoff
NBCNews.com
But in a softening of tone, Secretary of State Rex Tillerson said Tuesday that the U.S. is open to talks without preconditions. ?It's not realistic to say we're only going to talk if you come to the table ready to give up your program, they have too ...
North Koreans mark 6th anniversary of Kim Jong Il's deathFox News

all 98 news articles »
Birmingham crash: Six dead in 'horrific' smash - BBC News

The Guardian

Birmingham crash: Six dead in 'horrific' smash
BBC News
Six people have been killed in a "horrific" crash in Birmingham. Three vehicles were involved in the accident on Belgrave Middleway, near Edgbaston, at about 01:00 GMT. Three men in one car died at the scene and a fourth is critically injured in ...
Birmingham road accident leaves six people deadThe Guardian
'Harrowing' multi-vehicle crash leaves six dead in BirminghamSky News

all 66 news articles »
MSNBC's Chris Matthews was reprimanded over comments about woman in 1999 - NBCNews.com

NBCNews.com

MSNBC's Chris Matthews was reprimanded over comments about woman in 1999
NBCNews.com
MSNBC host Chris Matthews was accused of inappropriate jokes and comments about a female employee in 1999 and the woman was paid separation compensation, a spokesperson for the cable network confirmed Saturday. The Daily Caller, citing two sources ...

and more »
Pope Francis: fake and sensationalised news 'a very serious sin' - The Guardian

The Guardian

Pope Francis: fake and sensationalised news 'a very serious sin'
The Guardian
Pope Francis has criticised journalists who dredge up old scandals and sensationalise the news, saying it's a ?very serious sin? that hurts all involved. Francis, who plans to dedicate his upcoming annual communications message to ?fake news?, told ...

Police beef up Sen. Cory Booker's security following death threat - ABC News

Police beef up Sen. Cory Booker's security following death threat
ABC News
Sen. Cory Booker testifies during the second day of a confirmation hearing for Attorney General-designate, Sen. Jeff Sessions, before the Senate Judiciary Committee on Capitol Hill in Washington, D.C., Jan. 11, 2017.more +. 0 Shares. Email. Police in ...

and more »

MORE RESOURCES:

Warning: MagpieRSS: Failed to parse RSS file. (syntax error at line 1, column 0) in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/magpierss/rss_fetch.inc on line 238
Site:

Warning: Invalid argument supplied for foreach() in /home2/jsteiner/public_html/scholarlyarticles.org/spam-blocker/inc/rss.php on line 9
Site: spam blocker - Google News
This RSS feed URL is deprecated
This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news Spam is - The Outline

The Outline

Spam is
The Outline
For a while, spam ? unsolicited bulk messages sent for commercial or fraudulent purposes ? seemed to be fading away. The 2003 CAN-SPAM Act mandated unsubscribe links in email marketing campaigns and criminalized attempts to hide the sender's identity ...

How to Stop Spam Calls Before You Kill Your Phone - Southeast Missourian (blog)

How to Stop Spam Calls Before You Kill Your Phone
Southeast Missourian (blog)
Get your Landline an Inbound Call Blocker. If you are using a landline, you have fewer options in battling spam calls. While your carrier can also help you block calls from unwanted numbers, they can't entirely prevent scammers. The best way to go is ...

WatchGuard Firebox M5600 review - IT PRO

IT PRO

WatchGuard Firebox M5600 review
IT PRO
Mail security is handled by the spamBlocker service, and to use it we set up the POP3 proxy to tag messages classed as spam, suspect and bulk. It's very effective: in live tests of other WatchGuard appliances, we've seen spam detection rates of 97-98 ...

Return of Necurs botnet brings new ransomware threat - Help Net Security

Help Net Security

Return of Necurs botnet brings new ransomware threat
Help Net Security
Researchers found that hackers were using Necurs, considered to be the largest spam botnet in the world, to distribute the relatively new Scarab ransomware that was first seen in June 2017. The Necurs botnet started mass distribution of Scarab during ...

Stop spam calls with the best call- and text-blocker apps - Digital Trends

Digital Trends

Stop spam calls with the best call- and text-blocker apps
Digital Trends
Unwanted calls and texts are a serious problem. There were almost 30 billion robocalls in the U.S. last year according to YouMail research. Thankfully, the Federal Communications Commission is acting to try and protect consumers from unwanted ...

Hiya raises $18M to build business search and spam blocking into native smartphone dialer app - GeekWire

GeekWire

Hiya raises $18M to build business search and spam blocking into native smartphone dialer app
GeekWire
Founded last year after spinning out from Whitepages, Hiya is known for its caller ID and spam blocking app ? formerly known as Whitepages Caller ID ? that helps provide users with more information about incoming calls to their smartphone. It also ...

and more »
Necurs Botnet Returns to Top 10 Malware List - Infosecurity Magazine

Infosecurity Magazine

Necurs Botnet Returns to Top 10 Malware List
Infosecurity Magazine
The Necurs botnet has increased in prevalence since the US Thanksgiving holiday, as cyber-criminals use it to distribute a new form of ransomware, according to Check Point's latest Global Threat Impact Index. Over Thanksgiving, hackers were found using ...

T-Mobile's spam call blocking is coming to MetroPCS - The Verge

The Verge

T-Mobile's spam call blocking is coming to MetroPCS
The Verge
Back in March, T-Mobile introduced Scam ID and Scam Block, two features that help users avoid annoying robocalls, and now it's bringing the service to MetroPCS. Starting tomorrow, all MetroPCS users will automatically get Scam ID, the feature that ...
MetroPCS Fights Spam Calls Through Scam ID, Scam BlockAndroid Headlines
MetroPCS customers get Scam ID and Scam Block this weekTmoNews
Scam ID and Scam Block coming to Metro PCS to stop scammers from calling your iPhone9to5Mac
T-Mobile Newsroom
all 19 news articles »
Verizon is charging for anti-spam features T-Mobile and AT&T give away for free - The Verge

The Verge

Verizon is charging for anti-spam features T-Mobile and AT&T give away for free
The Verge
The Federal Communications Commission even passed a new rule proposal back in March that is expected to give more call blocking power to carriers to cut down on massive robocall complaints, far and away the number one complaint lodged with the FCC ...

and more »
How to Block SMS Spam on iOS 11 - NDTV

NDTV

How to Block SMS Spam on iOS 11
NDTV
SMS spam is something that plagues pretty much everyone who has an Indian SIM card, and even those who live abroad. Be it frauds, asking you to buy shares in companies to double your wealth, or just your neighbourhood store sending promotional messages ...

Home | Site Map | Resource Links