Home > Linux > Filtering using Squid

Filtering using Squid

Typical office set-up always have an internet connection.  Even stores have one and they are fast!  Thanks to wireless internet.  But, in a office of 30 or more employees with 1MBPS connection, you will have a different experience.  That’s because some of your users are downloading mp3’s, movies, etc while working.  Those downloads will slow down your internet not to mention the internet advertisement being displayed in most of the website that your users visited.

Hit google and research! <ding!> I found a solution – Proxy Server. Proxy’s main function is to cache the most visited web sites.  But wait, proxy is not only to cache most visited websites, it can do filtering too.

Did I say filtering?  Yes, you’re right filtering.  I know, most of users will hate you for doing this.  They’ll feel that they’re being suppressed of right to access the internet and everything.  Ha! ha! ha! I know I’ve been in your shoes.

But, What can you do? Your network is slowing down and you need to speed up the sending/receiving of email, the important websites for research.  Managers needs to communicate and do research.  Besides, the company needs to make money, right?  Otherwise, the company cannot pay your salaries, bla bla bla.

As the System or Network Administrator you need to speed up the internet connection without increasing the bandwidth ‘coz bigger bandwidth costs money.

It’s time to install SQUID proxy server!

There are number of proxy server on the internet why squid? Squid has been around for years and it’s stable and fast but most of all you can configure it according to your need. (whisper:  most important, it’s my personal favorite!)

Let’s dig in into business.

Below is the basic configuration to run squid proxy but, we are not interested in that configuration, we are more interested in filtering  of websites and internet advertisement, slowing down downloads and stuff.

#General Setup

acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
http_port 3128 transparent
icp_port  0
request_body_max_size 10240 KB
cache_mem 64 MB
cache_replacement_policy heap LFUDA
cache_dir ufs /opt/squid/cache 6000 16 256
access_log /var/log/squid/access.log squid

#Replace “server_hostname” with the hostname of your Ubuntu machine
visible_hostname <server name>

refresh_pattern -i .deb$ 0 50% 28800
refresh_pattern -i .rpm$ 0 50% 28800
refresh_pattern -i .tgz$ 0 50% 28800
refresh_pattern -i .exe$ 0 50% 28800
refresh_pattern -i .cab$ 0 50% 28800
refresh_pattern -i .zip$ 0 50% 28800
refresh_pattern -i .rar$ 0 50% 28800
refresh_pattern -i .arj$ 0 50% 28800
refresh_pattern -i .jpg$ 0 50% 28800
refresh_pattern -i .gif$ 0 50% 28800
refresh_pattern -i .bmp$ 0 50% 28800
refresh_pattern -i .mov$ 0 50% 28800
refresh_pattern -i .avi$ 0 50% 28800
refresh_pattern -i .mpg$ 0 50% 28800
refresh_pattern -i .mpeg$ 0 50% 28800
refresh_pattern -i .wmv$ 0 50% 28800
refresh_pattern -i .mp3$ 0 50% 28800
refresh_pattern -i .wav$ 0 50% 28800
refresh_pattern -i .bin$ 0 50% 129600

# caching TTL and DNS
negative_ttl 1 minutes
positive_dns_ttl 15 hours
negative_dns_ttl 1 minutes
half_closed_clients off

acl manager proto cache_object
acl CONNECT method CONNECT
acl PURGE method PURGE

Now that you have the basic configuration we will add the Access Lists.

You have to define your IP networks

acl Servers src 192.168.1.0/24
acl Production src 192.168.2.0/24
acl back-office src 192.168.3.0/24
acl sales src 192.168.4.0/24
acl Management src 192.168.5.0/24

Define the web sites that you want to deny and give full speed and most important slow down the downloads.

acl fullspeed dstdomain “/etc/squid/fullspeed.dat”
acl DeniedSites dstdomain “/etc/squid/deniedsites.dat”
acl slow_it_down url_regex -i “/etc/squid/slow_this_down”

Access Lists for ftp

# acl www_ports src 80 443
acl ftp_ports src 21
acl localhost src 127.0.0.1/32
acl manager proto cache_object
acl CONNECT method CONNECT
acl PURGE method PURGE

Define who can and don’t access the internet

http_access allow manager localhost our_network
http_access allow manager localhost
http_access allow servers back-office sales management
http_access allow PURGE
http_access allow localhost
http_access deny manager
http_access deny PURGE
http_access deny DeniedSites

Define the FTP account for anonymous FTP download.  Change domain for your domain name

ftp_user Squid@<domain>.com
ftp_passive off

Now, for serious stuff.

We don’t want to delay our Servers and management’s traffic.  Don’t forget to include your IP Address.

delay_parameters -1/-1 means that they have no limit in using the internet.  That includes you😉

delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow cubic-server fullspeed management networkadmin

For the Second delay pool.
we want to delay downloading files mentioned in slow_it_down.

The numbers here are values in bytes;

40000/500000 = 40 kbps download speed and 25 MB bucket for the network
15000/250000 = 15kbps download speed with 25 MB bucket for each user

after downloaded files exceed about 250000 bytes, they will continue to download at about 5000 bytes/s

delay_class 2 2
delay_parameters 2 40000/500000 15000/250000
delay_access 2 allow slow_it_down production back-office sales

That’s it.  Reload squid’s configuration.

# service squid reload

or

# /etc/init.d/squid reload

After configuring your proxy, it’s time to define the denied websites, etc.

create a file named deniedsites in /etc/squid

# vi /etc/squid/deniedsites

and add the sites that you want to deny.

.youtube.com
.facebook.com
.friendster.com
.video.msn.com
.megavideo.com
.doubleclick.net
.bannerconnect.net
.ads.clicksor.com
.ads.overclock.net
.ads.ozonemedia.co.in
.statcounter.com
.207.net
.2mdn.net
.advertising.com
.atdmt.com
.atwola.com
.intellitxt.com
.kanoodle.com
c.live.com
.msads.net
.ads1.msn.com
.rad.msn.com
c.msn.com
.pointroll.com
.revsci.net
.llnwd.net
.ugamsolutions.com
.zedo.com
sg.adserver.yahoo.com
pagead2.googlesyndication.com
.blip.tv
.dailymotion.com

Wait a minute.  Did I just include youtube, facebook and friendster?  Now, that’s a serious problem.  I will have a lot of question from the users.  Yes, you will hehehe and be ready to answer all of them or send out an IT Advisory that the company’s internet is slowing down bla bla bla.  You need to be creative on this🙂

The rests are internet advertising sites.

As for downloading of files, you have to define the extension name of the files that you want to slow down.

.ftp
.mp3
.vqf
.rpm
.zip
.avi
.mpeg
.mpe
.mpg
.qt
.ram
.rm
.raw
.wav
.mov

Now, define the web sites that can be downloaded very fast

.google.com
.yahoo.com
.<your domain>.com

NOTE:  Be on guard always, users might get back on you on what you’ve done on the internet hehehe

Categories: Linux
  1. StevenAB
    July 23, 2010 at 4:20 pm

    Great post!😀

  2. November 21, 2013 at 9:36 pm

    Hi there, I discovered your blog via Google while
    searching for a similar subject, your site came up, it seems good.
    I’ve bookmarked it in my google bookmarks.
    Hi there, simply turned into aware of your blog via Google, and found that it is truly informative.
    I’m gonna watch out for brussels. I’ll appreciate should you continue this in future.
    Lots of folks will probably be benefited out of your writing.

    Cheers!

  3. February 14, 2014 at 8:42 am

    This excellent website truly has all of the information and facts
    I needed about this subject and didn’t know who to ask.

  4. February 16, 2014 at 3:28 pm

    I drop a comment when I especially enjoy a article on a website or if I have something to contribute to the conversation.
    It’s a result of the sincerness displayed in the post I read.
    And after this article Filtering using Squid | Andoy lang’s Blog.
    I was actually moved enough to drop a thought😉 I actually do
    have some questions for you if it’s okay. Could it be simply me or does it look like some of these
    responses come across like left by brain dead folks?😛
    And, if you are writing on other places, I’d like to follow anything new you have to post.
    Would you list every one of your shared pages like your linkedin
    profile, Facebook page or twitter feed?

  5. February 25, 2014 at 4:06 am

    Thɑnks νerү ոice blog!

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: