MOHON BANTUAN SQUID

Diskusi tentang Ubuntu Server baik webserver, database server, samba server dan service lainnya serta jaringan menggunakan Sistem Operasi Ubuntu.
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

MOHON BANTUAN SQUID

Postby rangga_mone » 28 Feb 2013, 20:26

mohon pencerahan dunk Mastah... cara supaya cache squid ubuntu kita di clear otomatis saat restara/reboot gmn ya???? thx
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 28 Feb 2013, 20:59

Chache yang tersimpan di memory, otomatis terhapus saat reboot. Sedangkan yang di HDD, apa tdk sayang kalau di purge (clear) ??
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 28 Feb 2013, 23:51

hehehe... maaf telat reply, jadi gini Mas Pragola, saya punya squid itu tiap 2 hari macet karena full, padahal cache nya 18gb, dulu si sampe sebulan baru penuh tapi tiba2 skr lbh cepat... stlh di clear cache baru bisa jalan lagi... nah saat clear cache pun ga sampe bersih masih ada 44% yang ga bisa terhapus.. ko bisa gitu ya Mas? mohon pencerahannya... bingung juga ngadepin ini
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 01 Mar 2013, 00:13

Berarti opsi

Code: Select all

cache_swap_low ##
cache_swap_high ##
tdk jalan mas. Kalau opsi itu jalan chache dir tdk akan penuh.
dari cache yang berukuran 18 GB, berapa yang anda gunakan ? untuk hasil terbaik gunakan hanya +/- 70 % untuk cache
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 01 Mar 2013, 00:41

sebenarnya cahe saya 26GB, maksud saya maksimalnya itu 18GB,, oww... btw bole dijelaskan sedikit Mas Pragola tentang opsi yang Mas berikan itu? diletakkan di squid.conf kah?
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 01 Mar 2013, 00:48

Boleh lihat squid.conf mas ? mungkin ada yang bisa kami bantu u/ benah2 :)
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 01 Mar 2013, 00:56

bole dengan senang hati... tapi bsk mas... hehehe, di kantor.. gpp kah Mas Pragola?
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 01 Mar 2013, 01:00

Insya Allah :)
User avatar
mallink
Posts: 86
Joined: 24 Feb 2013, 23:50
Location: makassar
Contact:

Re: MOHON BANTUAN SQUID

Postby mallink » 02 Mar 2013, 00:08

ane juga mas udah jalan proxynya tapi lambat amat mas.

mohon pencerahannya.?
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 11:27

##############################################################
#config by Iwinduarta
##############################################################
http_port 3128 transparent
server_http11 on
icp_port 0
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_directory /usr/share/lusca/errors/English
icon_directory /usr/share/lusca/icons
visible_hostname iwinduarta.net
cache_mgr admin@localhost
access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log none
logfile_rotate 5
shutdown_lifetime 10 seconds
cache_mem 8 MB
maximum_object_size_in_memory 32 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA

#Cache dir
cache_dir aufs /cache 20500 48 256

#DNS Menggunkan unbound
dns_nameservers 127.0.0.1

dns_testnames 127.0.0.1
minimum_object_size 128 bytes
maximum_object_size 128000 KB
offline_mode off
cache_swap_low 98
cache_swap_high 99
# Setup some default acls
acl all src 0.0.0.0/0.0.0.0
acl localhost src 127.0.0.1/255.255.255.255
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 81 3128 1025-65535
acl sslports port 443 563 81
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT
acl snmppublic snmp_community public
acl apache rep_header Server ^Apache

#Localnet
acl iwinduarta src 192.168.3.0/24 # LAN jaringan
acl iwindgame src 192.168.0.0/24 # LAN Game

#start storeurl
##############################################################
acl store_rewrite_list dstdomain .google.com yahoo.com facebook.com
acl store_rewrite_list dstdomain .fbcdn.net .youtube.com .googlevideo.com .gstatic.com .bp.blogspot.com
acl store_rewrite_list dstdomain .4shared.com .twimg.com .ytimg.com .orkut.com
acl store_rewrite_list dstdomain .ggpht.com .media.tumblr.com .photobucket.com
acl store_rewrite_list dstdomain .avast.com .avg.com .vimeo.com .v.mccont.com .blip.tv
acl store_rewrite_list dstdomain .flashvideo.globo.com .terra.com
acl store_rewrite_list dstdomain .wpc.porntube.com .redtubefiles.com .xvideos.com .phncdn.com .pornhub.com
acl store_rewrite_list dstdomain .catalog.video.msn.com .mais.uol.com.br
acl store_rewrite_list dstdomain .videobb.com .sourceforge.net
acl store_rewrite_list_url url_regex (firefox|speedtest|iavs5x|iavs4x)


#cache content

acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$

acl dontrewrite url_regex yimg.com redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\?
acl getmethod method GET
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)


acl dontrewrite url_regex (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]*
acl dontrewrite url_regex \.(php|asp|aspx|jsp|cgi|js)\?
acl dontrewrite url_regex threadless.*\.jpg\?r=
storeurl_access deny dontrewrite
storeurl_access deny !getmethod


storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_url
storeurl_access deny all

storeurl_access deny all
storeurl_rewrite_children 1
#end storeurl acl

############################################################################################################################w
broken_vary_encoding allow apache
upgrade_http0.9 deny shoutcast
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

# Allow local network(s) on interface(s)
http_access allow iwinduarta
http_access allow iwindgame

# Default block all to be sure
http_access deny all

#REFRESH PATTERN TUNNING
##############################################################
refresh_pattern .zynga.com.*\.(swf|jpg|gif|png|mp3) 129600 999999% 129600 override-expire ignore-reload store-stale



## tes 2

# BINARY FILE
refresh_pattern -i \.(htm|html|asp|xml|class|css|js|swf|ico|cur|ani|jpg|jpeg|bmp|png|cdr|txt|gif|dll) 43200 99% 43200 ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|exe|msi|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar|deb|rpm|bin) 43200 99% 43200 ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?) 129600 999999% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(3gp|7z|ace|asx|divx|dvr-ms|ram|inc|cab|qt) 43200 99% 43200 ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob) 43200 99% 43200 ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 129600 999999% 129600 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv) 43200 99% 43200 ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale

# URL REGEX PARAMETER
refresh_pattern imeem.*\.flv 0 0% 0 override-lastmod override-expire store-stale
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280 90% 161280 ignore-reload store-stale
refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 129600 999999% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 129600 999999% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
refresh_pattern \.(ico|video-stats) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod negative-ttl=10080 store-stale
refresh_pattern \.etology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
refresh_pattern galleries\.video(\?|sz) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
refresh_pattern brazzers\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
refresh_pattern \.adtology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth store-stale negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth negative-ttl=10080 store-stale
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 129600 999999% 129600 override-expire ignore-reload ignore-private store-stale negative-ttl=10080
refresh_pattern ytimg\.com.*\.jpg 129600 999999% 129600 override-expire ignore-reload store-stale
refresh_pattern images\.friendster\.com.*\.(png|gif|jpg|bmp|gif|ico|cur|ani) 129600 999999% 129600 override-expire ignore-reload store-stale
refresh_pattern garena\.com 129600 999999% 129600 override-expire reload-into-ims store-stale
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 override-expire ignore-reload store-stale
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 129600 999999% 129600 ignore-no-cache override-expire override-lastmod store-stale
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 reload-into-ims override-expire ignore-private store-stale
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\. 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale
refresh_pattern ^http:\/\/www.onemanga.com.*\/ 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale
refresh_pattern ^http:\/\/www.porntube.com.*\/ 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern (avgate|avira).*(idx|gz)$ 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern kaspersky.*\.avc$ 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern kaspersky 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern update.nai.com/.*\.(gem|zip|mcs) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale

# WINDOWS UPDATE
refresh_pattern windowsupdate.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern download.microsoft.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale

# FACEBOOK IMAGES
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif) 129600 999999% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3) 129600 999999% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern static\.ak\.fbcdn\.net*\.(jpg|gif|png) 129600 999999% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png) 129600 999999% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store store-stale

# IIX BANNER
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern ^http:\/\/openx.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale

#--------------------------------------------#
# TAG DOWNLOAD
#--------------------------------------------#
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store store-stale ignore-auth

refresh_pattern (cgi-bin|\?) 0 0% 0
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern ^ftp: 10080 95% 43200 override-lastmod reload-into-ims store-stale
refresh_pattern . 180 95% 43200 override-lastmod reload-into-ims store-stale

## tes2 end

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
##############################################################

#flow qos mark HIT
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

#SNMP enable for MRTG
snmp_port 3401
snmp_access allow snmppublic all

##############################################################

header_access Accept-Encoding deny all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
#range_offset_limit 50 KB
read_timeout 30 minutes
client_lifetime 6 hours
negative_ttl 30 seconds
positive_dns_ttl 6 hours
negative_dns_ttl 60 seconds
pconn_timeout 15 seconds
request_timeout 1 minute
store_avg_object_size 13 KB
log_icp_queries off
ipcache_size 16384
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384
memory_pools off
forwarded_for off
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 11:28

sbenarnya sy bukn engineer jaringan, cuma sekedar hobby aja si, lmyn lah bisa sy terapkan dikantor... hhehehehe

mohon bantuannya Mas Pragola Pati.....
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 02 Mar 2013, 12:20

@rangga_mone
Menggunakan RAM berapa mas ?
storeurl_rewrite_program kok tdk ada ?
HDD yang untuk cache terpisah dengan system atau cuma 1 HDD untuk system dan cache ?
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 14:32

RAM cuma 512mb x 2
storeurl_rewrite_program ini untuk apa si Mas?
HDD cuma 1 80GB, sy mo pake terpisah tp menurut teori dr inet katanya klo ram cm 1GB itu cuma bisa ngehandle sekitar 25% dari ram, jadi maksumal 18gb hdd aja... bisakah cache kita kasi 80GB?
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 14:34

kalo sya pake
storeurl_rewrite_program /etc/squid/storeurl.pl

yang isinya

#!/usr/bin/perl
# $Rev$
# by chudy_fernandez@yahoo.com
# Updates at http://wiki.squid-cache.org/ConfigExamp ... Discussion
$|=1;
while () {
@X = split;
# $X[1] =~ s/&sig=.*//;
$x = $X[0] . " ";
$_ = $X[1];
$u = $X[1];


# compatibility for old cached get_video?video_id
if (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "\n";

# youtube HD itag=22
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=22).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $2 . "&" . $3 . "\n";

# youtube Normal screen always HD itag 35, Normal screen never HD itag 34, itag=18 <--normal?
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $3 . "\n";

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

# spicific servers starts here....
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $3 . $5 . "\n";
#...spicific servers end here.

#photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
print $x . "http://photos.ak.fbcdn.net/" . $1 . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#generic http://variable.domain.com/path/filename. or "exte" with or withour "?"
} elsif (m/^http:\/\/(.*)([^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{3,4})(\?.*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-A-]+[0-9-]+)|(.*cdn.*)|(.*cache.*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# generic http://variable.domain.com/...
} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
print $x . "http://cdn." . $3 . "." . $4 . "/" . $5 . "\n";

# spicific extention that ends with ?
} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|on2)\?(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "." . $3 . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $_ . "\n";
}
}


jalan ga Mas?
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 02 Mar 2013, 15:10

@ Rangga_Mone
Coba anda edit postingan anda yang ada squid.conf dan storeurl.pl itu dan masukkan ke dalam "spoiler" biar tdk melebar ke mana-mana layar monitor-nya
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 02 Mar 2013, 16:32

@rangga_mone
Kayaknya (kayaknya lho mas), problem-nya squid anda sudah merambah ke swap.
1.Maka yang bisa anda lakukan adalah optimasi penggunaan RAM (1 GB) tsb =
Tampilkan

Code: Select all

##############################################################
#config by Iwinduarta
##############################################################
http_port 3128 transparent
server_http11 on
icp_port 0
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_directory /usr/share/lusca/errors/English
icon_directory /usr/share/lusca/icons
visible_hostname iwinduarta.net
cache_mgr admin@localhost
access_log /var/log/squid/access.log   
cache_log /var/log/squid/cache.log   >>> cache_log /dev/null
cache_store_log none         >>> cache_store_log /dev/null
logfile_rotate 5         >>> logfile_rotate 1
shutdown_lifetime 10 seconds
cache_mem 8 MB            >>> cache_mem 6 MB
maximum_object_size_in_memory 32 KB   >>> maximum_object_size_in_memory 0 KB   
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA

#Cache dir
cache_dir aufs /cache 20500 48 256   >>> cache_dir aufs /proxy_1 15000 32 256

#DNS Menggunkan unbound
dns_nameservers 127.0.0.1

dns_testnames 127.0.0.1         >>> #dns_testnames 127.0.0.1
minimum_object_size 128 bytes      >>> minimum_object_size 8 bytes
maximum_object_size 128000 KB      >>> maximum_object_size 700 MB
offline_mode off
cache_swap_low 98
cache_swap_high 99

# Setup some default acls
acl all src 0.0.0.0/0.0.0.0
acl localhost src 127.0.0.1/255.255.255.255
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 81 3128 1025-65535
acl sslports port 443 563 81
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT
acl snmppublic snmp_community public   >>> #acl snmppublic snmp_community public
acl apache rep_header Server ^Apache

#Localnet
acl iwinduarta src 192.168.3.0/24 # LAN jaringan
acl iwindgame src 192.168.0.0/24 # LAN Game

#start storeurl
##############################################################
acl store_rewrite_list dstdomain .google.com yahoo.com facebook.com
acl store_rewrite_list dstdomain .fbcdn.net .youtube.com .googlevideo.com .gstatic.com .bp.blogspot.com
acl store_rewrite_list dstdomain .4shared.com .twimg.com .ytimg.com .orkut.com
acl store_rewrite_list dstdomain .ggpht.com .media.tumblr.com .photobucket.com
acl store_rewrite_list dstdomain .avast.com .avg.com .vimeo.com .v.mccont.com .blip.tv
acl store_rewrite_list dstdomain .flashvideo.globo.com .terra.com
acl store_rewrite_list dstdomain .wpc.porntube.com .redtubefiles.com .xvideos.com .phncdn.com .pornhub.com
acl store_rewrite_list dstdomain .catalog.video.msn.com .mais.uol.com.br
acl store_rewrite_list dstdomain .videobb.com .sourceforge.net
acl store_rewrite_list_url url_regex (firefox|speedtest|iavs5x|iavs4x)


#cache content

acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$

acl dontrewrite url_regex yimg.com redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\?
acl getmethod method GET
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)


acl dontrewrite url_regex (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]*
acl dontrewrite url_regex \.(php|asp|aspx|jsp|cgi|js)\?
acl dontrewrite url_regex threadless.*\.jpg\?r=
storeurl_access deny dontrewrite
storeurl_access deny !getmethod


storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_url
storeurl_access deny all

storeurl_access deny all
storeurl_rewrite_children 1   >>>storeurl_rewrite_program /etc/squid/storeurl.pl
            >>>storeurl_rewrite_children 2
            >>>storeurl_rewrite_concurrency 10

            >>>include /etc/squid/refresh_pattern.conf

#end storeurl acl



############################################################################################################################w
broken_vary_encoding allow apache
upgrade_http0.9 deny shoutcast
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

# Allow local network(s) on interface(s)
http_access allow iwinduarta
http_access allow iwindgame

# Default block all to be sure
http_access deny all


#flow qos mark HIT
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

#SNMP enable for MRTG
snmp_port 3401            >>>#snmp_port 3401
snmp_access allow snmppublic all   >>>#snmp_access allow snmppublic all

##############################################################

header_access Accept-Encoding deny all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
#range_offset_limit 50 KB
read_timeout 30 minutes
client_lifetime 6 hours
negative_ttl 30 seconds
positive_dns_ttl 6 hours
negative_dns_ttl 60 seconds
pconn_timeout 15 seconds
request_timeout 1 minute
store_avg_object_size 13 KB
log_icp_queries off
ipcache_size 16384      >>>ipcache_size 1024
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384      >>>fqdncache_size 1024
memory_pools off
forwarded_for off
_________________________
---------------------------

2. Agar squid.conf anda tdk kepanjangan. Buat satu file sendiri untuk menampung reffresh_pattern dan storeurl.pl, tempatkan di /etc/squid

Code: Select all

cd /etc/squid
touch refresh_pattern.conf
nano refresh_pattern.conf   ### pastekan refresh_pattern anda di sini.
chown proxy:proxy refresh_pattern.conf
touch storeurl.pl
nano storeurl.pl      ### pastekan storeurl.pl anda di sini.
chown proxy:proxy storeurl.pl

3. Tambah HDD satu lagi khusus u/ cache_dir. Jadikan hanya satu partisi dengan voleme maks 20 GB dan sisanya tdk usah dipakai.
4. Tambah RAM sesuai kapasitas maksimal motherboard.
User avatar
mallink
Posts: 86
Joined: 24 Feb 2013, 23:50
Location: makassar
Contact:

Re: MOHON BANTUAN SQUID

Postby mallink » 02 Mar 2013, 16:35

mas squid ane dah jalan tapi ko download lambat gi mana pencerahannya mas.?
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 18:12

WOW.... :D

luar biasa, sepertinya harus install ulang ya Mas Pragola Pati... hehehehe... ntar sy laporkan perkembangan selanjutnya.... terimakasih bnyk Mas... luar biasa

btw, refresh pattern yang seperti

refresh_pattern \.(ico|video-stats) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod negative-ttl=10080 store-stale

gpp kah? soalnya saya bingung dengan 129600 999999% 129600, ini artinya apa ya Mas? apa ga cepat pnh cachenya? trus, klo kita mau setting si cache ga perlu di clear scr manual bisa kita setting dmna Mas? init.d atw ada perintah tambahan di squid.conf?

coba Mas Pragola dekat ya, pingin berguru banget euy...
User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Re: MOHON BANTUAN SQUID

Postby q_p » 02 Mar 2013, 19:00

Pengaturan refresh_pattern yang terlalu besar bisa mencegah squid menghapus cache meskipun cache_dir sudah mencapai batas dalam pengaturan cache_swap_high dan cache_swap_low. Akibatnya penuh, seperti yang anda keluhkan.
Prinsipnya, untuk situs yang frekuensi update-nya tinggi seperti situs berita dan antivirus pengaturan refresh_pattern jangan terlalu besar (exp = 1440 999999% 10080)
Tetapi itu semua tergantung dari perilaku client dalam ber-internet dalam jaringan anda, butuh penyesuaian dan proses (tdk bisa "sim salabim, jadilah squid yang top-markotop" )
cmiiw.
User avatar
rangga_mone
Posts: 46
Joined: 03 Dec 2012, 16:49
Location: Dompu, Indonesia
Contact:

Re: MOHON BANTUAN SQUID

Postby rangga_mone » 02 Mar 2013, 19:21

owh gitu ya... baiklah, akan sy rubah n lihat sedikit demi sedikit squid.conf nya, thx a lot sudah mau berbagi Mas Pragola :D

Return to “Ubuntu Server”

Who is online

Users browsing this forum: No registered users and 7 guests