[HOT POOLLING]Apakah Youtube Error Lagi ?

Diskusi tentang Ubuntu Server baik webserver, database server, samba server dan service lainnya serta jaringan menggunakan Sistem Operasi Ubuntu.
User avatar
riophone
Posts: 34
Joined: 21 Mar 2013, 21:40
Location: Medan, Indonesia

Re: [HOT POOLLING]Apakah Youtube Error Lagi ?

Post by riophone » 20 Jul 2013, 21:46

@Pragola_Pati
Kalau saya kendalanya tetap di autoupdate Google Chrome dan Mozilla Firefox walaupun sudah dimatikan fungsi autoupdatenya.
Kejadiannya kebetulan setelah salah satu warnet mati lampu, begitu semua pc nyala autoupdate google chrome otomatis jalan dan menghabiskan bandwidth sampai 4MB.
Saya sudah coba utak atik di storeurl tapi hasilnya tetap belum tercache.
Untuk sementara terpaksa saya blokir auto updatenya.

TCP_DENIED/403 2999 GET http://cache.pack.google.com/edgedl/upd ... eSetup.exe - NONE/- text/html


User avatar
Zest
Posts: 12
Joined: 13 Mar 2012, 17:24
Location: Madiun - Jatim - Indonesia
Contact:

Post by Zest » 20 Jul 2013, 21:50

Youtube dah lancar Gan...
Ga ada trouble...


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 20 Jul 2013, 22:22

riophone wrote:@Pragola_Pati
Kalau saya kendalanya tetap di autoupdate Google Chrome dan Mozilla Firefox walaupun sudah dimatikan fungsi autoupdatenya.
Kejadiannya kebetulan setelah salah satu warnet mati lampu, begitu semua pc nyala autoupdate google chrome otomatis jalan dan menghabiskan bandwidth sampai 4MB.
Saya sudah coba utak atik di storeurl tapi hasilnya tetap belum tercache.
Untuk sementara terpaksa saya blokir auto updatenya.

TCP_DENIED/403 2999 GET http://cache.pack.google.com/edgedl/upd ... eSetup.exe - NONE/- text/html
soreurl tidak usah disentuh dulu, coba anda tambahkan dalam squid.conf

Code: Select all

acl gopack dstdomain .pack.google.com
cache allow gopack
...
...
acl partial_dom dstdomain .pack.google.com
range_offset_limit -1 KB partial_dom
range_offset_limit 64 KB
...
...
refresh_patern mungkin juga harus ditambahi ...

Code: Select all

cat /var/log/squid/access.log | grep "pack.google.com"
tolong output dikopas di sini mas :)

u/ firefox tidak ada masalah kok =
1374335639.292 2724 192.168.0.100 TCP_REFRESH_HIT/200 12910394 GET http://download.cdn.mozilla.net/pub/moz ... artial.mar - DIRECT/118.98.37.8 application/octet-stream
1374335639.292 2724 192.168.0.100 TCP_REFRESH_HIT/200 12910394 GET http://download.cdn.mozilla.net/pub/moz ... artial.mar - DIRECT/118.98.37.8 application/octet-stream
@Zest
Syukurlah ... :)


User avatar
foolbaby
Posts: 7
Joined: 26 Mar 2013, 14:52

Post by foolbaby » 21 Jul 2013, 18:06

maaf mas dah lama gak cek lusca, apakah youtub udah gak harus pakai referer log lagi ya ? kok disaya kalao pakai referer log suka error video / sound kadang gak metching (meskipun hit )
thx


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Jul 2013, 18:47

Ada dua pendekatan mas, bisa pakai referer_log atau tidak pakai. Untuk amannya, tetap gunakan option referer_log pada saat ngupil meskipun tidak digunakan pada konfigurasi squid.


User avatar
foolbaby
Posts: 7
Joined: 26 Mar 2013, 14:52

Post by foolbaby » 21 Jul 2013, 18:53

oww bgtu ya buat jaga2 klo si youtube berubah lagi coding nya ?
thx om kalo bgtu :)


User avatar
dermovel
Posts: 20
Joined: 12 Jul 2013, 20:49

Post by dermovel » 21 Jul 2013, 22:19

Pragola_Pati wrote:
nanangcaesar wrote:Salam Kenal semuanya,
alhamdulillah setelah penuh jalan berliku akhirnya setelah mengikuti tutorial dari awal..its work!
Saya ikuti langkah di hal 38 dan berjalan normal sampai detik ini
Terima kasih, saya ikut senang :)
Mas Dermoval yang belum sukses, keep spirit mas Dermoval :)
proxy saya sering down mungkin karena sudah menggunakan swap yg berlebih hampir 50% dari jatah yg disediakan :hajarpc:

Alhasil saya uninstall ubuntu 10.10 dan saya install 10.04 LTS dan menggunakan tutorial dari page 1 dan di sambung di page 38 tanpa opsi "include" (squid.conf saya gabung jadi 1).

Sekilas saya coba2 tidak semua file youtube bisa tercache...any idea??? :crazy:


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Jul 2013, 22:35

[quote:dermovel]proxy saya sering down mungkin karena sudah menggunakan swap yg berlebih hampir 50% dari jatah yg disediakan hajarpc

Alhasil saya uninstall Ubuntu 10.10 dan saya install 10.04 LTS dan menggunakan tutorial dari page 1 dan di sambung di page 38 tanpa opsi "include" (squid.conf saya gabung jadi 1).
...
[/quote]
Swappiness tidak berhubungan dengan versi Ubuntu Server. Untuk Server produksi, sebaiknya memang menggunakan LTS (10.04 atau 12.04). RAM tekor dan proxy merambah ke SWAP, pernah di bahas di sini = http://ubuntu-indonesia.com/forums/ubbt ... Post138856 dan ini kutipannya =
Pragola_Pati wrote:Beberapa skenario =
  • Coba anda re-compile dengan menghilangkan opsi2 yang tidak diperlukan. Misalnya jika tidak ada rencana menggunkan snmp maka '--disable-snmp'.
  • Tambah RAM sesuai dengan kapasitas-maks slot motherboard
  • sesuiakan dir-cache dengan kapasitas ram yang terpasang
  • menggunakan memory-allocation alternatif
  • Solusi sementara bisa di lihat pada trit ini = http://ubuntu-indonesia.com/forums/ubbt ... Post135934
[quote:dermovel]
Sekilas saya coba2 tidak semua file youtube bisa tercache...any idea??? crazy[/quote] Coba anda gunakan storeurl versi terakhir.


User avatar
dermovel
Posts: 20
Joined: 12 Jul 2013, 20:49

Post by dermovel » 21 Jul 2013, 23:03

[quote:Pragola_Pati] Coba anda gunakan storeurl versi terakhir. [/quote]

maksudnya yang terakhir benar yang ini
https://code.google.com/p/tempat-sampah ... svn61&r=61
atau yang lainnya??

sorry mas numpang tanya sekalian, saya buka facebook lalu cek di access.log nya squid koq tidak ada ya... apakah krn facebook menggunakan format https??


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Jul 2013, 23:22

Iya betul, perhatikan kelompok baris paling atas yang berwarna merah. Di situlah kuncinya.
Iya betul, facebook sudah pakai https. Sebagai tanda protes, dveloper storeurl dalam ling yang anda maksud tadi "sudah menutup akun facebook-nya" :) Bentar lagi saya ... giliran anda kapan ? :)


User avatar
gakguna
Posts: 4
Joined: 13 Jul 2013, 15:14

Post by gakguna » 23 Jul 2013, 10:43

[spoiler]

Code: Select all

root@proxy:~# /usr/local/squid/sbin/squid -NDd1 &
[1] 3400
root@proxy:~# 2013/07/23 09:10:07| WARNING: '192.168.10.0/255.255.255.0' is a subnetwork of '192.168.10.0/255.255.255.0'
2013/07/23 09:10:07| WARNING: because of this '192.168.10.0/255.255.255.0' is ignored to keep splay tree searching predictable
2013/07/23 09:10:07| WARNING: You should probably remove '192.168.10.0/255.255.255.0' from the ACL named 'rojil'
2013/07/23 09:10:07| WARNING: Very large maximum_object_size_in_memory settings can have negative impact on performance
2013/07/23 09:10:07| Starting Squid Cache version LUSCA_HEAD-r14809 for i686-pc-linux-gnu...
2013/07/23 09:10:07| Process ID 3400
2013/07/23 09:10:07| NOTICE: Could not increase the number of filedescriptors
2013/07/23 09:10:07| With 1024 file descriptors available
2013/07/23 09:10:07| Using poll for the IO loop
2013/07/23 09:10:07| Adding nameserver 203.130.208.18 from squid.conf
2013/07/23 09:10:07| Adding nameserver 103.247.216.202 from squid.conf
2013/07/23 09:10:07| logfileOpen: opening log /var/log/squid/access.log
2013/07/23 09:10:07| Unlinkd pipe opened on FD 10
2013/07/23 09:10:07| Swap maxSize 37888000 + 1048576 KB, estimated 2995121 objects
2013/07/23 09:10:07| Target number of buckets: 149756
2013/07/23 09:10:07| Using 262144 Store buckets
2013/07/23 09:10:07| Max Mem  size: 1048576 KB
2013/07/23 09:10:07| Max Swap size: 37888000 KB
2013/07/23 09:10:07| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2013/07/23 09:10:07| logfileOpen: opening log /var/log/squid/store.log
2013/07/23 09:10:07| AUFS: /cache: log '/cache/swap.state' opened on FD 12
2013/07/23 09:10:07| AUFS: /cache: tmp log /cache/swap.state.new opened on FD 12
2013/07/23 09:10:07| Rebuilding storage in /cache (CLEAN)
2013/07/23 09:10:07| Using Least Load store dir selection
2013/07/23 09:10:07| Current Directory is /root
2013/07/23 09:10:07| Loaded Icons.
2013/07/23 09:10:07| commBind: Cannot bind socket FD 14 family 2 to 0.0.0.0 port 3128: (98) Address already in use
FATAL: Cannot open HTTP Port
[/spoiler]

:) mohon bantuan nya ya semua :)


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 23 Jul 2013, 16:55

gakguna wrote:root@proxy:~# /usr/local/squid/sbin/squid -NDd1 &
[1] 3400
root@proxy:~# 2013/07/23 09:10:07| WARNING: '192.168.10.0/255.255.255.0' is a subnetwork of '192.168.10.0/255.255.255.0'
2013/07/23 09:10:07| WARNING: because of this '192.168.10.0/255.255.255.0' is ignored to keep splay tree searching predictable
2013/07/23 09:10:07| WARNING: You should probably remove '192.168.10.0/255.255.255.0' from the ACL named 'rojil'
2013/07/23 09:10:07| WARNING: Very large maximum_object_size_in_memory settings can have negative impact on performance
2013/07/23 09:10:07| Starting Squid Cache version LUSCA_HEAD-r14809 for i686-pc-linux-gnu...
2013/07/23 09:10:07| Process ID 3400
2013/07/23 09:10:07| NOTICE: Could not increase the number of filedescriptors
2013/07/23 09:10:07| With 1024 file descriptors available
2013/07/23 09:10:07| Using poll for the IO loop
2013/07/23 09:10:07| Adding nameserver 203.130.208.18 from squid.conf
2013/07/23 09:10:07| Adding nameserver 103.247.216.202 from squid.conf
2013/07/23 09:10:07| logfileOpen: opening log /var/log/squid/access.log
2013/07/23 09:10:07| Unlinkd pipe opened on FD 10
2013/07/23 09:10:07| Swap maxSize 37888000 + 1048576 KB, estimated 2995121 objects
2013/07/23 09:10:07| Target number of buckets: 149756
2013/07/23 09:10:07| Using 262144 Store buckets
2013/07/23 09:10:07| Max Mem size: 1048576 KB
2013/07/23 09:10:07| Max Swap size: 37888000 KB
2013/07/23 09:10:07| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2013/07/23 09:10:07| logfileOpen: opening log /var/log/squid/store.log
2013/07/23 09:10:07| AUFS: /cache: log '/cache/swap.state' opened on FD 12
2013/07/23 09:10:07| AUFS: /cache: tmp log /cache/swap.state.new opened on FD 12
2013/07/23 09:10:07| Rebuilding storage in /cache (CLEAN)
2013/07/23 09:10:07| Using Least Load store dir selection
2013/07/23 09:10:07| Current Directory is /root
2013/07/23 09:10:07| Loaded Icons.
2013/07/23 09:10:07| commBind: Cannot bind socket FD 14 family 2 to 0.0.0.0 port 3128: (98) Address already in use
FATAL: Cannot open HTTP Port

:) mohon bantuan nya ya semua :)
======================================================

coba mas ganti port 3128-nya jadi 8000 atau 8080 ....


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 23 Jul 2013, 19:48

Atau,
Cek dulu siapa yang pakai port 3128.


User avatar
McFeter
Posts: 1
Joined: 14 Feb 2012, 15:26

Post by McFeter » 24 Jul 2013, 08:51

Numpang tanya access.log squid dah 6.8GB bisakan kita split berdasarkan bulan atau tahun. trims


User avatar
gakguna
Posts: 4
Joined: 13 Jul 2013, 15:14

Post by gakguna » 24 Jul 2013, 09:44

Pragola_Pati wrote:Atau,
Cek dulu siapa yang pakai port 3128.
port 3128 nya transparent mas...

ini mas saya gelar nih squid.conf nya

[spoiler]

Code: Select all

# SQUID LUSCA TEST CONFIG FILE

# Email: ojhee.pasti@yahoo.com
# Web  : rojilsquid.com

# PORT and Transparent Option
http_port 3128 transparent

#server_http11 on
#icp_port 0

# Cache Directory , modify it according to your system.
# but first create directory in root by mkdir /cache1
# and then issue this command  chown proxy:proxy /cache
# [for ubuntu user is proxy, in Fedora user is SQUID]
# I have set 500 MB for caching reserved just for caching ,
# adjust it according to your need.
# My recommendation is to have one cache_dir per drive. zzz

#store_dir_select_algorithm round-robin
cache_dir aufs /cache1 37000 16 256
cache_dir aufs /cache2 41000 16 256
cache_dir aufs /cache3 47000 16 256

cache_replacement_policy heap LFUDA
memory_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following
#emulate_httpd_log on
#logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
#log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
#logfile_rotate 14
#debug_options ALL,1

cache_access_log /var/log/lusca/access.log
cache_log /var/log/lusca/cache.log
cache_store_log /var/log/lusca/store.log

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 203.130.208.18 103.247.216.202

ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl rojil src 192.168.5.0/24
#acl rojil src 192.168.5.3
acl rojil src 192.168.10.0/24

acl DENYCACHE urlpath_regex \.(ini|tsl|ui|lst|inf|ver|patch|md5|cfg|lst|list|rsc|log|conf|dbd|db)$ 
acl DENYCACHE urlpath_regex (notice.html|afs.dat|dat.asp|patchinfo.xml|version.list|iepngfix.htc)$
acl DENYCACHE urlpath_regex (xtrapva.dll|xtrap.xt|updates.txt|patchlist.txt|gs_service_login.php)$
acl DENYCACHE urlpath_regex (pointblank.css|login_form.css|form.css)$
acl DENYCACHE urlpath_regex (Loader|gamenotice|login|sources|captcha|notice|reset)
acl DENYCACHE urlpath_regex  ^http://www.gemscool.com/registration/*
acl yt url_regex -i (youtube|google).*videoplayback.*range=[0-9]*

acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT

# YouTube
#acl yt url_regex -i (youtube|google).*videoplayback.*range=[0-9]*
#deny_info http://jaringanwarnet.com/adb/block.png yt
#http_access deny yt

#acl RW url_regex \/speedtest\/.*\.(jpg|txt|php).*
#url_rewrite_access allow RW

#url_rewrite_program /etc/lusca/rewriter.pl
#url_rewrite_children 7
#url_rewrite_concurrency 10
#redirector_bypass on

#============================#
## http_access
#============================#

http_access deny !rojil
#http_access deny yt
cache deny DENYCACHE
http_access allow rojil
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
http_reply_access allow all
icp_access allow all

#==========================
# Administrative Parameters
#==========================

# I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr ojheepasti 
visible_hostname rojilsquid 

#=============
# ACCELERATOR
#=============
half_closed_clients off
quick_abort_min 0 KB
quick_abort_max 0 KB
vary_ignore_expire on
reload_into_ims on
log_fqdn off
memory_pools off
cache_swap_low 98
cache_swap_high 99
max_filedescriptors 65536
fqdncache_size 16384
retry_on_error on
offline_mode off
pipeline_prefetch on

# If you want to hide your proxy machine from being detected at various site use following
#via off

#============================================
# OPTIONS WHICH AFFECT THE CACHE SIZE / zaib
#============================================

# If you have 4GB memory in Squid box, we will use formula of 1/3
# You can adjust it according to your need. IF squid is taking too much of RAM
# Then decrease it to 128 MB or even less.

cache_mem 1024 MB
minimum_object_size 0 bytes
maximum_object_size 1024 MB
maximum_object_size_in_memory 512 MB

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG
#============================================================$
#acl snmppublic snmp_community gl
#snmp_port 3401
#snmp_access allow snmppublic all
#snmp_access allow all

#============================================================
# ZPH , To enable cache content to be delivered at full lan speed,
# To bypass the queue at MT.
#============================================================
zph_mode tos
zph_local 0x04
zph_parent 0
zph_option 136

include /etc/lusca/pattern.conf

# nano /etc/sysctl.conf "net.ipv4.ip_forward=1"
# nano /etc/rc.local 
# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.6.1:3128
# iptables -t nat -A POSTROUTING -s 192.168.6.0/24 -d 0/0 -j MASQUERADE

# zph tos

# iptables -t mangle -A OUTPUT -m tos --tos Maximize-Reliability -j MARK --set-mark 0x4 
# iptables -t mangle -A OUTPUT -m tos --tos 0x4 -j MARK --set-mark 0x4 
# iptables -t mangle -A FORWARD -m tos --tos 0x4 -j MARK --set-mark 0x4 
# iptables -t mangle -A POSTROUTING -m tos --tos 0x4 -j MARK --set-mark 0x4 

# mangle icmp pinging
# iptables\A0\ADt\A0mangle\A0\ADA\A0PREROUTING \ADp\A0icmp -j MARK --set-mark 2
# iptables -t mangle -A POSTROUTING \ADp\A0icmp -j MARK --set-mark 2

#chown proxy:proxy /cache1
#chmod -R  777 /cache1

#touch /etc/lusca/storeurl.pl
#chmod +x /etc/lusca/storeurl.pl
[/spoiler]

mohon bantuannya :)


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Jul 2013, 19:05

@gakguna
coba anda stopdulu squid-nya baru anda jalan squid mode-debug

Code: Select all

service squid stop
usr/local/squid/sbin/squid -NDd1 &


User avatar
dermovel
Posts: 20
Joined: 12 Jul 2013, 20:49

Post by dermovel » 24 Jul 2013, 23:13

ini squidnya bermasalah atau saya yang bermasalah ya..... :crazy:

setiap web yang sudah pernah saya buka, saya coba buka kembali dengan browser yang berbeda, hasilnya blank :hajarpc:


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 00:47

dermovel wrote:ini squidnya bermasalah atau saya yang bermasalah ya..... :crazy:

setiap web yang sudah pernah saya buka, saya coba buka kembali dengan browser yang berbeda, hasilnya blank :hajarpc:
squid-nya mas :confused:


User avatar
Aira
Posts: 70
Joined: 22 Mar 2012, 21:20

Post by Aira » 25 Jul 2013, 01:16

Malam om semuanya lama tidak bertanya cuman bisa melihat2 saja...
Sekalinya muncul sya bertanya maaf yaa...
Sya punya kasus begini (Lihat di gambar "XML") kenapa ya begitu saja...
Kalo proxy di matikan tidak begitu...

Link-nya


[imgleft]http://s7.postimg.org/rbo1w75iz/errrorr.jpg[/img]


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 02:27

dermovel wrote:ini squidnya bermasalah atau saya yang bermasalah ya..... :crazy:

setiap web yang sudah pernah saya buka, saya coba buka kembali dengan browser yang berbeda, hasilnya blank :hajarpc:
dan juga untuk proxy yang sering down...
coba di cek lagi setting di mikrotik..apa sampean aktifkan tool netwatch? saya dulu pengalaman begitu kalo pake net watch kemudian saya disable
@mas pragola & @mas SJW thanks for sharing..pantesan FB nya wes hilang..:)


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 02:31

@Aira
Apakah anda menggunakan acl untuk banned file extention atau regex ? dan xml masuk di dalamnya ? soalnya link tsb saya coba buka, hasilnya tampil dengan wajar :)


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 02:37

@ALL
di squid 2.7 say coba menggunakan fitur Tproxy dengan --enable -tproxy..kegunaanya sendiri ndak tahu,
apa ada pengaruh jika fitur tersebut enable / disable? (saya pakai dengan single interface)


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 02:59

nanangcaesar wrote:@ALL
di squid 2.7 say coba menggunakan fitur Tproxy dengan --enable -tproxy..kegunaanya sendiri ndak tahu,
apa ada pengaruh jika fitur tersebut enable / disable? (saya pakai dengan single interface)
Maksud-nya Squid-2.7.STABLE9 ? Untuk bisa menggunakan fitur TProxy pada Squid-2.7.STABLE9 sepertinya harus melakukan patch kernel dan patch squid-2.7.STABLE9 (dari balabit). Coba anda cermati pada saat kompil, nantinya ada semacam error/warning yang berhubungan dengan patch tsb jika sebelumnya tdk melakukan patching.
cmiiw


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 03:26

sekarang saya sudah memakai fitur tproxy (saya compile awal,saya ambil dari "tempatsampahnya mas SJW" dan sekarang selama beberapa hari saya pakai, yang saya tanyakan apa perbedaan jika kita memakai Tproxy dan tdk pakai...krn saya tidak mersakan pengaruhnya mas...

root@starnet:~# squid -v
Squid Cache: Version 2.7.STABLE9
configure options: '-prefix=/usr' '-exec-prefix=/usr' '-bindir=/usr/sbin' '-sbindir=/usr/sbin' '-libexecdir=/usr/lib/squid' '-sysconfdir=/etc/squid' '-localstatedir=/var/spool/squid' '-datadir=/usr/share/squid' '--enable-gnuregex' '--enable-async-io=24' '--with-aufs-threads=24' '--with-pthreads' '--with-aio' '--with-dl' '--enable-storeio=aufs,null' '--enable-removal-policies=heap' '--enable-icmp' '--enable-delay-pools' '--disable-wccp' '--enable-snmp' '--enable-cache-digests' '--enable-default-err-language=English' '--enable-err-languages=English' '--disable-linux-netfilter' '--enable-linux-tproxy' '--disable-ident-lookups' '--with-maxfd=65535' '--enable-follow-x-forwarded-for' '--enable-large-cache-files' '--with-large-files' '--enable-referer-log' '--build=x86_64-linux-gnu' 'build_alias=x86_64-linux-gnu'
..ini yg saya pakai compile nya


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 04:18

Maksud saya bukan itu. Pada saat anda meng-kompil dan menjalan 'make && make install' akan ada pesan error/warning jika anda tdk melakukan patching kernel/squid. Semacam ini =

Code: Select all

WARNING: Cannot find TPROXY headers, you need to patch your kernel with the
tproxy package from:
 - lynx http://www.balabit.com/downloads/files/tproxy/
BTW, untuk bisa menjalankan fitur TProxy masih dibutuhkan langkah2 pengaturan tingkat lanjut (setting iproute2 & iptables untuk intercepting tproxy)


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 04:35

Pragola_Pati wrote:Maksud saya bukan itu. Pada saat anda meng-kompil dan menjalan 'make && make install' akan ada pesan error/warning jika anda tdk melakukan patching kernel/squid. Semacam ini =

Code: Select all

WARNING: Cannot find TPROXY headers, you need to patch your kernel with the
tproxy package from:
 - lynx http://www.balabit.com/downloads/files/tproxy/
BTW, untuk bisa menjalankan fitur TProxy masih dibutuhkan langkah2 pengaturan tingkat lanjut (setting iproute2 & iptables untuk intercepting tproxy)
kebetulan (mungkin)source yang saya ambil sudah terpatch dengan Tproxy sehingga waktu compile tidak ada error/warning seperti itu...
setelah compile baru melakukan langkah mangle di mikrotik dan di proxy (iptable dan startup)
kalo gak salah sebelumnya saya coba dulu squid tanpa tproxy sudah berjalan baik


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 04:51

Betul, sources Squid-2.7.STABLE9 sudah patched dengan tproxy. Bagaimana dengan kernel ubuntu server yang anda gunakan ? Apakah sudah ter-patch agar support dengan tproxy ?
Dalam kasus yang berbeda, misalnya saya ingin menggunakan file-system reiser4 untuk dir_cache, maka saya harus menyiapkan kernel-nya seperti dalam trit ini = http://ubuntu-indonesia.com/forums/ubbt ... Post138912


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 05:09

saya melakukan test dulu apakh support tproxy apa tidak di ubuntu server yang saya pakai skarang (Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-49-generic x86_64)) dengan perintah dan dengan hasil =

Code: Select all

cat /boot/config-`uname -r` | grep -E '(NF_CONNTRACK=|TPROXY|XT_MATCH_SOCKET|XT_TARGET_T PROXY)'
MATCH_SOCKET|XT_TARGET_T PROXY)'
CONFIG_NF_CONNTRACK=m
CONFIG_NETFILTER_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
yang dari info yang saya dapatkan berarti ubuntu support tproxy
kalau gak salah mas
Pragola_Pati wrote:Betul, sources Squid-2.7.STABLE9 sudah patched dengan tproxy. Bagaimana dengan kernel ubuntu server yang anda gunakan ? Apakah sudah ter-patch agar support dengan tproxy ?
Dalam kasus yang berbeda, misalnya saya ingin menggunakan file-system reiser4 untuk dir_cache, maka saya harus menyiapkan kernel-nya seperti dalam trit ini = http://ubuntu-indonesia.com/forums/ubbt ... Post138912
saya belum mampir ke threat ini yang mas


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 05:44

...oOo... pakai 12.04 tho, saya kira ... :)
Di atas tadi tanya kegunaannya ya ? Maksudnya kelebihannya jika dibandingkan menggunakan metoda tranparent ?
http://wiki.squid-cache.org/ConfigExamp ... WithTPROXY
http://wiki.squid-cache.org/Features/Tproxy4
http://www.balabit.com/downloads/files/ ... README.txt


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 05:54

Pragola_Pati wrote:...oOo... pakai 12.04 tho, saya kira ... :)
Di atas tadi tanya kegunaannya ya ? Maksudnya kelebihannya jika dibandingkan menggunakan metoda tranparent ?
Ya mas...itu lebih mengena pertanyaanya..kelebihan tproxy dengan tanpa tproxy tranparant....baru kenal dgn ubuntu dan squid


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 07:27

Saya tidak tahu persis-nya. Pada 'metode transparent', kita masih bisa menembus brikade (misalnya filtering) dengan menggunakan proxy-luar (semacam vpn, hidemyash/ip) pada browser.
Tapi kalau TProxy, bagaimana hasilnya ? tolong anda coba ya, saya tunggu hasilnya :)


User avatar
nanangcaesar
Posts: 9
Joined: 27 Apr 2013, 00:13
Location: Surabaya

Post by nanangcaesar » 25 Jul 2013, 07:57

Pragola_Pati wrote:Saya tidak tahu persis-nya. Pada 'metode transparent', kita masih bisa menembus brikade (misalnya filtering) dengan menggunakan proxy-luar (semacam vpn, hidemyash/ip) pada browser.
Tapi kalau TProxy, bagaimana hasilnya ? tolong anda coba ya, saya tunggu hasilnya :)
Sepertinya saya sudah mulai bingung..hehe..di situs hidemyass,bisa masuk dan memakai fasilitas disana sebagian(yang kena dampak internet positif), yang lainnya(sbobet,ttmini(situs judi) kadang hanya bisa tampil halaman login tapi ndak bisa dilanjutkan
BTW di squid.conf saya pakai begini (sepertinya salah)

Code: Select all

http_port 3128 transparent
http_port 3129 tproxy
server_http11 on
icp_port 0
mohon koreksinya mas


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 25 Jul 2013, 08:30

Pengaturannya ada pada link yang saya lampirkan pada reply di atas :)


User avatar
dermovel
Posts: 20
Joined: 12 Jul 2013, 20:49

Post by dermovel » 25 Jul 2013, 14:33

nanangcaesar wrote:
dermovel wrote:ini squidnya bermasalah atau saya yang bermasalah ya..... :crazy:

setiap web yang sudah pernah saya buka, saya coba buka kembali dengan browser yang berbeda, hasilnya blank :hajarpc:
dan juga untuk proxy yang sering down...
coba di cek lagi setting di mikrotik..apa sampean aktifkan tool netwatch? saya dulu pengalaman begitu kalo pake net watch kemudian saya disable
@mas pragola & @mas SJW thanks for sharing..pantesan FB nya wes hilang..:)
Saya emang aktifkan netwatch di mikrotik saya... Coba saya disable dulu..

Masalah di squid saya ketika saya buka web yang sudah pernah dibuka, lalu di coba di browser laen dan hang klo web yg belum pernah saya buka lancar.... sudah solve....

Ini karena zph local saya rubah dari 0x04 ke 0x30 lalu hang, kira2 arti dari nilai tsb apa ya??

, saya pernah baca2 jika zph local 0x30 Maka seting dscp tos mikrotik=12 (ada formula perhitungannya tp saya lupa) klo zph local 0x04 dscp tos mikrotik jadi berapa ya...????


User avatar
Bandi_Shippuden
Posts: 62
Joined: 04 May 2012, 12:06
Location: Pekanbaru, Indonesia
Contact:

Post by Bandi_Shippuden » 26 Jul 2013, 12:30

kalo mau cache video selain youtube g mana ya ???
di tempat ane kok gak bisa ...
misalnya kayak video di sini
http://www.fungeta.com/2013/07/naruto-s ... nesia.html


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 31 Jul 2013, 21:10

@ all

akhir2 ini kok saya download game PC lewat cdn.adf.ly selalu ga keluar link ya???? apakah teman2 juga mengalaminya ????


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 31 Jul 2013, 22:05

cah_clacap wrote:@ all

akhir2 ini kok saya download game PC lewat cdn.adf.ly selalu ga keluar link ya???? apakah teman2 juga mengalaminya ????
Ra mudeng saya, bisa kasih skringsut-nya mas ?


User avatar
Bandi_Shippuden
Posts: 62
Joined: 04 May 2012, 12:06
Location: Pekanbaru, Indonesia
Contact:

Post by Bandi_Shippuden » 04 Sep 2013, 00:34

udah ada yg dapet solusi belum biar youtube gak pindah2 resolusi ...


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 04 Sep 2013, 09:32

bandi16 wrote:udah ada yg dapet solusi belum biar youtube gak pindah2 resolusi ...
Kayaknya yang ini tidak masalah mas dan pernah di bahas pada halaman2 sebelum-nya.


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 05 Sep 2013, 17:44

@Mas Pragola Pati n Mas Syaifuddin JW dan semua masta di sini, ane minta bantuan nich

ane nyetting jaringan punya teman, kebetulan konfigurasinya begini
IP dari modem 192.168.1.1 (asli)
IP LOKAL / LAN 100.50.25.0/24
IP Proxy 192.168.60.2


apakah antara IP lokal dan Proxy yang beda subnet tersebut bisa berkomunikasi???
Maksudnya apakah setingan pakai IP tersebut dapat berjalan???
Ataukah saya harus mengganti IP Proxy misal 100.50.26.2??

Mohon beri saya pencerahan mas semua .... Terima kasih sebelumnya atas koment yg diberikan


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 05 Sep 2013, 17:48

cah_clacap wrote:@Mas Pragola Pati n Mas Syaifuddin JW dan semua masta di sini, ane minta bantuan nich

ane nyetting jaringan punya teman, kebetulan konfigurasinya begini
IP dari modem 192.168.1.1 (asli)
IP LOKAL / LAN 100.50.25.0/24
IP Proxy 192.168.60.2


apakah antara IP lokal dan Proxy yang beda subnet tersebut bisa berkomunikasi???
Maksudnya apakah setingan pakai IP tersebut dapat berjalan???
Ataukah saya harus mengganti IP Proxy misal 100.50.26.2??

Mohon beri saya pencerahan mas semua .... Terima kasih sebelumnya atas koment yg diberikan

=========================================

mengingat ip lan dan ip proxy beda kelas kok saya bingung ya ??? mohon bantuannya mas semua ... ,makasih


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 05 Sep 2013, 19:41

Arahkan semua request dari klient ke ip-proxy, dengan iptables (kalau ubuntu server sebagai router-gateway). Kalau menggunakan mikrotik ?


User avatar
SyaifuddinJW
Posts: 152
Joined: 22 Feb 2010, 10:59
Contact:

Post by SyaifuddinJW » 07 Sep 2013, 10:45

tentu saja ip ip tersebut dapat berhubungan apabila ada sebuah router, baik itu router dari linux, bsd, cisco atau apapun yg berfungsi sebagai router
kalau saya pribadi lebih prefer untuk bermain routing dari pada bermain NAT.
sedangkan untuk membelokkan trafficnya ya bermain redirec ataupun dstnat tergantung apa dan dimana bermainnya


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 12 Sep 2013, 23:15

Pragola_Pati wrote:Arahkan semua request dari klient ke ip-proxy, dengan iptables (kalau ubuntu server sebagai router-gateway). Kalau menggunakan mikrotik ?
+++++++++++++++++++++++++++

OK bos makasih bgt bantuannya


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 12 Sep 2013, 23:15

SyaifuddinJW wrote:tentu saja ip ip tersebut dapat berhubungan apabila ada sebuah router, baik itu router dari linux, bsd, cisco atau apapun yg berfungsi sebagai router
kalau saya pribadi lebih prefer untuk bermain routing dari pada bermain NAT.
sedangkan untuk membelokkan trafficnya ya bermain redirec ataupun dstnat tergantung apa dan dimana bermainnya
+++++++++++++++++++++++++++++


Ok bos makasih bgt bantuannya


User avatar
sigaret
Posts: 29
Joined: 04 Feb 2013, 13:08
Location: Bandung
Contact:

Post by sigaret » 18 Sep 2013, 13:47

cah_clacap wrote:
Pragola_Pati wrote:Arahkan semua request dari klient ke ip-proxy, dengan iptables (kalau ubuntu server sebagai router-gateway). Kalau menggunakan mikrotik ?
+++++++++++++++++++++++++++

OK bos makasih bgt bantuannya
kalo menggunakan mikrotik tinggal port forwading
exp:
ip fir nat ad chain=dstnat action=dst-nat src-address 100.50.25.0/24 to-address=192.168.60.2 to-port=3128(port default proxy) protocol=tcp dst-port=80

dengan catatan
network 192.168.60.0/24, 100.50.25.0/24 udah di NAT
posisi proxy berada di bawah router

maaf kalo ada salah salah kata newbie juga


User avatar
Bandi_Shippuden
Posts: 62
Joined: 04 May 2012, 12:06
Location: Pekanbaru, Indonesia
Contact:

Post by Bandi_Shippuden » 19 Sep 2013, 00:01

Pragola_Pati wrote:
bandi16 wrote:udah ada yg dapet solusi belum biar youtube gak pindah2 resolusi ...
Kayaknya yang ini tidak masalah mas dan pernah di bahas pada halaman2 sebelum-nya.
di tempat saya kalo proxy di gabung sama mikrotik yutube sering macet di tengah2 dan gak mau jalan lagi, gara2 resolusi berubah2 ...


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 19 Sep 2013, 00:06

Masak sih berubah sendiri, tanpa campurtangan user ? Coba pada semua kompi client,
  1. anda setting pada high_performance
  2. gunakan clear_type untuk meng-haluskan tampilan font pada lcd
  3. update browser, flash-player dan java


User avatar
Bandi_Shippuden
Posts: 62
Joined: 04 May 2012, 12:06
Location: Pekanbaru, Indonesia
Contact:

Post by Bandi_Shippuden » 05 Oct 2013, 12:59

Pragola_Pati wrote:Masak sih berubah sendiri, tanpa campurtangan user ? Coba pada semua kompi client,
  1. anda setting pada high_performance
  2. gunakan clear_type untuk meng-haluskan tampilan font pada lcd
  3. update browser, flash-player dan java
2 dan 3 udah saya lakuin ... yg 1 belum paham ....

ow ya mas kemaren keliling saya nemu ini ...

saya kutip sedikit kata2 dia ...
[30th September,2013, > working on Dailymotion , partial caching , under test phases]

As we all know that Mikrotik web proxy is a basic proxy package , suitable for basic caching , but its not possible to do the caching of Dynamic Contents, youtube videos and many other contents. To accomplish this task you have to add SQUID proxy server , and route all http traffic from mikrotik to squid, then configure squid 2.7 stable9 with storeurl URL rewrite.

I wrote an easy guide regarding squid compilation via its source package and its configuration for caching video and other contents. It�s working good till date, and caching most of the videos including youtube, and many others. I have listed few web sites that are caching good.

Usually, this sort of caching is possible with commercial products only, like an addon of squid name CACHEVIDEO, or hardware products, But with some R&D , hit and trials, & some working configs examples, the caching worked. Please be aware that i have not reinvented the wheel, the method is out there since few years, but with some modifications and updates, its now working very good. I am still working on it to improve it. This config have few junk entries that are outdated or not required any more. You should do some research on it, like few directives on refresh patterns that are not supported in 2.7

This guide is actually its a collection of squid and storeurl configuration guides, picked up from multiple public and shared resources.Its not 100% perfect, but it does it job at some good level :) / zaib

Blow is a quick reference guide for Squid 2.7 stable9 installation on Ubuntu ver 10.4 (or 12) with youtube & few other contents caching support. (or any linux flavor with Squid 2.7, because storeurl method is supported in squid 2.7 only ).
ini squid.conf nya ...
# SQUID 2.7 Stable9 Configuration FILE with updated STOREURL.PL [jz]
# For youtube and others contents caching, tested and working as of 27th September, 2013 [jz]
# Tested with Ubuntu 10.4 + Squid 2.7 STABLE 9 & Ubuntu 12.4 and Squid 2.7/S9 [jz]
# Various contents copied from multiple public shared sources, personnel configs, hits and trial, vc etc
# It do have lot of junk / un-necessary entries, so remove them if not required.
# Syed Jahanzaib / http://aacable.wordpress.com [jz]
# Email: aacable@hotamil.com

# PORT and Transparent Option [jz]
http_port 8080 transparent
server_http11 on
icp_port 0

# Cache Directory , modify it according to your system. [jz]
# but first create directory in root by mkdir /cache1
# and then issue this command chown proxy:proxy /cache1
# [for ubuntu user is proxy, in Fedora user is SQUID]
# I have set 200 GB for caching reserved just for caching ,
# adjust it according to your need.
# My recommendation is to have one cache_dir per drive. /zaib

# Using 200 GB in this example per drive
store_dir_select_algorithm round-robin
cache_dir aufs /cache-1 204800 16 256

# Cache Replacement Policies [jz]
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following [jz]
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %h %ru
#access_log /var/log/squid/yt.log squid1 yutub
acl redirec urlpath_regex -i .*&redirect_counter=1&cms_redirect=yes
acl redirec urlpath_regex -i .*&ir=1&rr=12
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*redirect_counter=1.*$
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*cms_redirect=yes.*$
storeurl_access deny reddeny

#--------------------------------------------------------#
# REFRESH PATTERN [jz] UPDATED: 27th September, 2013 [jz]
#--------------------------------------------------------#
# 1 year = 525600 mins, 1 month = 43800 mins
################################################

# General FILES including Apple Updates i.e ipsw [jz]
refresh_pattern -i \.(ipsw|pkg|dmg|asp|xml|ashx|class|css|js|swf|ico|cur|ani|jpg|jpeg|bmp|png|cdr|txt|gif|dll) 43200 99% 43200 ignore-reload ignore-no-cache ignore-no-store override-expire override-lastmod reload-into-ims store-stale
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|exe|ashx|msi|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar|deb|rpm|bin) 43200 99% 43200 ignore-reload ignore-no-cache override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?) 129600 999999% 129600 ignore-reload ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(3gp|7z|ace|asx|divx|dvr-ms|ram|inc|cab|qt) 43200 99% 43200 ignore-reload ignore-no-cache override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob) 43200 99% 43200 ignore-reload ignore-no-cache override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 129600 999999% 129600 ignore-reload ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv) 43200 99% 43200 ignore-reload ignore-no-cache override-expire override-lastmod reload-into-ims

# Various URL REGEX PARAMETER [jz]
refresh_pattern imeem.*\.flv 0 0% 0 override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280 90% 161280 ignore-reload
refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?|\.mov?|\.mp4?) 129600 999999% 129600 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?|\.mov?|\.mp4?) 129600 999999% 129600 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern \.SQUIDINTERNAL 229600 50% 865000 override-expire override-lastmod ignore-reload
refresh_pattern \.(ico|video-stats) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth override-lastmod negative-ttl=10080
refresh_pattern \.etology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache
refresh_pattern galleries\.video(\?|sz) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache
refresh_pattern yahoo\? 129600 999999% 129600 ignore-reload override-expire reload-into-ims
refresh_pattern \.adtology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache
refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-private override-expire ignore-reload ignore-auth negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth negative-ttl=10080
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 129600 999999% 129600 override-expire ignore-reload ignore-private negative-ttl=10080
refresh_pattern ytimg\.com.*\.jpg 129600 999999% 129600 override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif|jpg|bmp|gif|ico|cur|ani) 129600 999999% 129600 ignore-reload override-expire ignore-reload
refresh_pattern garena\.com 129600 999999% 129600 override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 ignore-reload override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 129600 999999% 129600 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\. 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/ 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-reload override-expire
refresh_pattern ^http:\/\/www.porntube.com.*\/ 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-reload override-expire

# Google Search Images
refresh_pattern (gstatic|diggstatic)\.com/.* 43200 9999% 43200 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern ^http:\/\/\.*\.gstatic\.com\/(.*) 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale

# ANTI VIRUS [jz]
refresh_pattern guru.avg.com/.*\.(bin) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern (avgate|avira).*(idx|gz) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern update.nai.com/.*\.(gem|zip|mcs) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern symantecliveupdate.com.*\(zip|exe) 43200 999999% 43200 ignore-no-cache ignore-no-store ignore-reload reload-into-ims store-stale
refresh_pattern kaspersky.*\.avc 129600 999999% 129600 ignore-reload store-stale
refresh_pattern kaspersky 1440 50% 161280 ignore-no-cache store-stale

# WINDOWS UPDATE [jz]
refresh_pattern windowsupdate.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-reload reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-reload reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe) 43200 999999% 129600 ignore-no-cache ignore-reload reload-into-ims

# FACEBOOK IMAGES & few GAMES, You can add more entries as required [jz]
refresh_pattern ((facebook.com)|(69.171.237.16)).*\.(jpg|jpeg|flv|swf|png|gif) 129600 999999% 129600 ignore-reload override-expire ignore-no-cache
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|ashx|mp3) 4600 999999% 4600 ignore-reload override-expire ignore-no-cache
refresh_pattern static\.ak\.fbcdn\.net*\.(jpg|gif|png) 4600 999999% 4600 ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png) 4600 999999% 4600 ignore-reload override-expire ignore-no-cache

refresh_pattern ^http:\/\/apps.facebook.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store store-stale
refresh_pattern -i \.zynga.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern -i \.farmville.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern -i \.ninjasaga.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern -i \.mafiawars.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern -i \.crowdstar.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern -i \.popcap.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-must-revalidate store-stale

# IIX BANNER, I am sure not required much, remove them if not required [jz]
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache

#-----------------#
# General Pattern
#-----------------#
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-auth

refresh_pattern ^ftp: 1440 100% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 8260 100% 161280 reload-into-ims

########################################################################
## MORE REFRESH PATTERN SETTINGS (including video cache config too) [jz]
########################################################################

acl dontrewrite url_regex -i \.(youtube|googlevideo)\.com\/(get_video|videoplayback).*(begin|start)\=[1-9\-]*.*
acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id) \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf)\?
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex streamate.doublepimp.com.*\.js\? .fbcdn.net \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.* yield$
acl store_rewrite_list_domain_CDN url_regex (cbk|mt|khm|mlt|tbn)[0-9]?.google\.co(m|\.uk|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http://(.*?)/windowsupdate\?
acl store_rewrite_list_domain_CDN url_regex photos-[a-z].ak.fbcdn.net
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/([a-z])[0-9]?(\.gstatic\.com|\.wikimapia\.org)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/download[0-9]{3}.avast.com/iavs5x/
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/dnl-[0-9]{2}.geo.kaspersky.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[1-4].bp.blogspot.com
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.fid\?.*\&start= \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|psf|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.*
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|zip|flv|wm(a|v)|3gp|psf|mp(4|3)|exe|msi|avi|(mp(e?g|a|e|1|2|3|4))|cab|exe)
acl videocache_allow_url url_regex -i \.googlevideo\.com\/videoplayback \.googlevideo\.com\/videoplay \.googlevideo\.com\/get_video\?
acl videocache_allow_url url_regex -i \.google\.com\/videoplayback \.google\.com\/videoplay \.google\.com\/get_video\?
acl videocache_allow_url url_regex -i \.google\.[a-z][a-z]\/videoplayback \.google\.[a-z][a-z]\/videoplay \.google\.[a-z][a-z]\/get_video\?
acl videocache_allow_url url_regex -i proxy[a-z0-9\-][a-z0-9][a-z0-9][a-z0-9]?\.dailymotion\.com\/
acl videocache_allow_url url_regex -i \.vimeo\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \.youporn\.com\/(.*)\.flv
acl videocache_allow_url url_regex -i \.msn\.com\.edgesuite\.net\/(.*)\.flv
acl videocache_allow_url url_regex -i \.tube8\.com\/(.*)\.(flv|3gp)
acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
acl videocache_allow_url url_regex -i \.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
acl videocache_allow_url url_regex -i \.apniisp\.com\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
acl videocache_allow_url url_regex -i \.break\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i redtube\.com\/(.*)\.flv
acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/
acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\.xtube\.com\/(.*)flv
acl videocache_allow_url url_regex -i bitcast\.vimeo\.com\/vimeo\/videos\/
acl videocache_allow_url url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \.files\.youporn\.com\/(.*)\/flv\/
acl videocache_allow_url url_regex -i \.msn\.com\.edgesuite\.net\/(.*)\.flv
acl videocache_allow_url url_regex -i media[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/ mobile[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/ www\.tube8\.com\/(.*)\/
acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
acl videocache_allow_url url_regex -i \.video[a-z0-9]?[a-z0-9]?\.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram)
acl videocache_allow_url url_regex -i video\.break\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i \.xvideos\.com\/videos\/flv\/(.*)\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv)
acl videocache_allow_url url_regex -i msn\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i msn\.(.*)\.(com|net)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i msnbc\.(.*)\.(com|net)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.blip\.tv\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url url_regex -i \.blip\.tv\/(.*)filename
acl videocache_allow_url url_regex -i \.break\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn\.turner\.com\/(.*)/(.*)\.(flv)
acl videocache_allow_url url_regex -i \.dailymotion\.com\/video\/[a-z0-9]{5,9}_?(.*)
acl videocache_allow_url url_regex -i proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.mccont\.com\/ItemFiles\/(.*)?\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.vimeo\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.amazonaws\.com\/(.*)\.vimeo\.com(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i v\.imwx\.com\/v\/wxcom\/[a-zA-Z0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)\?(.*)videoId=[0-9]+&
acl videocache_allow_url url_regex -i c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/
acl videocache_allow_url url_regex -i c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.extremetube\.phncdn\.com\/(.*)\/[a-zA-Z0-9_-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url url_regex -i \.hardsextube\.com\/videothumbs
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.keezmovies\.phncdn\.com\/(.*)\/[0-9a-zA-Z_\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.keezmovies\.com\/(.*)\/[0-9a-zA-Z_\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i nyc-v[a-z0-9]?[a-z0-9]?[a-z0-9]?\.pornhub\.com\/(.*)/videos/[0-9]{3}/[0-9]{3}/[0-9]{3}/[0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.video\.pornhub\.phncdn\.com\/videos/(.*)/[0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i video(.*)\.redtubefiles\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.slutload-media\.com\/(.*)\/[a-zA-Z0-9_.-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.spankwire\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.spankwire\.phncdn\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.tube8\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.xtube\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url url_regex -i \.xtube\.com\/(.*)(Thumb|videowall)
acl videocache_allow_url url_regex -i \.xvideos\.com\/videos\/flv\/(.*)\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i \.public\.youporn\.phncdn\.com\/(.*)\/[a-zA-Z0-9_-]+\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)

acl videocache_allow_dom dstdomain .mccont.com .metacafe.com .redtube.com .dailymotion.com .fbcdn.net
acl videocache_deny_dom dstdomain .download.youporn.com .static.blip.tv
acl dontrewrite url_regex redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]*
acl getmethod method GET
storeurl_access deny !getmethod
storeurl_access deny dontrewrite
storeurl_access allow videocache_allow_url
storeurl_access allow videocache_allow_dom
storeurl_access allow store_rewrite_list_domain_CDN
storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
storeurl_access deny all

# Load SOTREURL.PL REWRITE PROGRAM
storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 15
storeurl_rewrite_concurrency 999

acl store_rewrite_list urlpath_regex -i \/(get_video\?|videodownload\?|videoplayback.*id)
acl store_rewrite_list urlpath_regex -i \.flv$ \.mp3$ \.mov$ \.mp4$ \.swf$ \
storeurl_access allow store_rewrite_list
storeurl_access deny all
dan ini storeurl.pl nya
#!/usr/bin/perl
# This script is NOT written or modified by me, I only copy pasted it from the internet.
# It was First originally Written by chudy_fernandez@yahoo.com
# & Have been modified by various persons over the net to fix/add various functions.
# Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
# th30nly @comstuff.net a.k.a invisible_theater , and possibly other people too.
# For more info, http://wiki.squid-cache.org/ConfigExamp ... nt/YouTube
# Syed Jahanzaib / aacable@hotmail.com
# http://aacable.wordpress.com/2012/01/19 ... oreurl-pl/
#####################
#### REFERENCES ##### http://www2.fh-lausitz.de/launic/comp/m ... t_youtube/
#####################
#####################

#!/usr/bin/perl
## storeurl.pl-130411-0.05
## s. 130411.howto.squid.youtube.html
## CHANGES
## 130411 hl+mf
## - fix youtube loop redir
## - rel. 0.05
## 130409 hl
## - add usleep
## 120726 hl
## - add printtimenow
## 120725 hl
## - remove html-tags
## - add debug
## - disable unused rules
## - rel. 0.04
## 120130 aacable
## - fix match variable.domain.com
## http://aacable.wordpress.com/2012/01/30 ... er-solved/
## 120111 aacable
## http://aacable.wordpress.com/2012/01/11 ... tik-queue/
#
#### notes
## - config (s. 130411.howto.squid.youtube.html)
## squid.conf:storeurl_rewrite_program /etc/squid/storeurl.pl
## - test
## $ cat squid.diag.youtube.url | awk '{print "0",$7,"10.0.0.1/- - GET - myip=10.0.0.2 myport=3128"}' | ./storeurl.pl

#### var
use IO::File;
$|=1;
STDOUT->autoflush(1);
$debug=1; ## recommended:0
$bypassallrules=0; ## recommended:0
$sucks=""; ## unused
$sucks="sucks" if ($debug>=1);
$timenow="";
$printtimenow=1; ## print timenow: 0|1
my $logfile = '/tmp/storeurl.log';

open my $logfh, '>>', $logfile
or die "Couldn't open $logfile for appending: $!\n" if $debug;
$logfh->autoflush(1) if $debug;

#### main
## in : 0 http://server/path?var 192.168.5.10/- - GET - myip=192.168.1.224 myport=3128
while () {
$timenow=time()." " if ($printtimenow);
print $logfh "$timenow"."in : $_" if ($debug>=1);
chop; ## strip eol
@X = split;
$x = $X[0]; ## 0
$u = $X[1]; ## url
$_ = $u; ## url

if ($bypassallrules){
$out="$u"; ## map 1:1

#youtube with range (YOUTUBE has split its videos into segments)
}elsif (m/(youtube|google).*videoplayback\?.*range/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@range = m/[&?](range=[^\&\s]*)/;
@begin = m/[&?](begin=[^\&\s]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTER ... &@redirect";
#sleep(1); ## delay loop

#youtube without range
}elsif (m/(youtube|google).*videoplayback\?/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTER ... &@redirect";
#sleep(1); ## delay loop

#speedtest
}elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
$out="http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "";

#mediafire
}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
$out="http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "";

#fileserve
}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
$out="http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "";

#filesonic
}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
$out="http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "";

#4shared
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3";

#4shared preview
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/$2";

#photos-X.ak.fbcdn.net where X a-z
}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
$out="http://photos" . $1 . "/" . $2 . $3 . "";

#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1 ."/". $2 . "";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
$out="http://" . $1 . $2 . "";

# compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "";
#sleep(1); ## delay loop

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
$out="http://www.google-analytics.com/__utm.gif";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
$out="http://" . $1 . $2 . $y . "";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
$out="http://" . $1 . "/" . $2 . "";

# spicific servers starts here....
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
$out="http://" . $1 . "";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
$out="http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
$out="http://" . $1 . "/SQUIDINTERNAL/" . $3 . "";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
$out="http://cdn." . $4 . $6 . "";
#...spicific servers end here.

#photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1 . "";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
$out="http://cdn.yimg.com//" . $3 . "";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
$out="http://cdn.yimg.com/" . $3 . "";

#for yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
$out="http://" . $y[0] . ".yimg.com/" . $y[1] . "";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
$out="http://squid-cdn-url//" . $2 . "." . $3 . "";

#generic http://variable.domain.com/path/filename. "ext" or "exte" with or withour "? or %"
} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

## generic http://variable.domain.com/...
#} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
#$out="http://cdn." . $3 . "." . $4 . "/" . $5 . "";

## spicific extention that ends with ?
#} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|rar|cab|amf|swf)(.*)/) {
#$out="http://" . $1 . "/" . $2 . "." . $3 . "";

## all that ends with ;
#} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
#$out="http://" . $1 . "/" . $2 . "";

} else {
$out="$u"; ##$X[2]="$sucks";
}
print $logfh "$timenow"."out: $x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n" if ($debug>=1);
print "$x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n";
}
close $logfh if ($debug);
sumbernya dari ini mas ...
aacable.wordpress.com


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 05 Oct 2013, 13:38

Iya blog-nya orang tersebut ramai, sampai akhirnya muncul yang seperti ini =
[spoiler]#!/usr/bin/perl
# This script is NOT written or modified by me, I only copy pasted it from the internet.
# It was First originally Written by chudy_fernandez@yahoo.com
# & Have been modified by various persons over the net to fix/add various functions.
# Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
# th30nly @comstuff.net a.k.a invisible_theater , and possibly other people too.
# For more info, http://wiki.squid-cache.org/ConfigExamp ... nt/YouTube
# Syed Jahanzaib / aacable@hotmail.com
# http://aacable.wordpress.com/2012/01/19 ... oreurl-pl/
#####################
#### REFERENCES ##### http://www2.fh-lausitz.de/launic/comp/m ... t_youtube/
#####################
#####################[/spoiler]
Lucunya lagi, pada bagian refresh_pattern ada "kompas dan kaskus". Masak ada client-nya (di negara tsb) yang baca kompas ?[spoiler]# IIX BANNER, I am sure not required much, remove them if not required [jz]
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache[/spoiler]


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 05 Oct 2013, 21:06

q_p wrote:Iya blog-nya orang tersebut ramai, sampai akhirnya muncul yang seperti ini =
[spoiler]#!/usr/bin/perl
# This script is NOT written or modified by me, I only copy pasted it from the internet.
# It was First originally Written by chudy_fernandez@yahoo.com
# & Have been modified by various persons over the net to fix/add various functions.
# Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
# th30nly @comstuff.net a.k.a invisible_theater , and possibly other people too.
# For more info, http://wiki.squid-cache.org/ConfigExamp ... nt/YouTube
# Syed Jahanzaib / aacable@hotmail.com
# http://aacable.wordpress.com/2012/01/19 ... oreurl-pl/
#####################
#### REFERENCES ##### http://www2.fh-lausitz.de/launic/comp/m ... t_youtube/
#####################
#####################[/spoiler]
Lucunya lagi, pada bagian refresh_pattern ada "kompas dan kaskus". Masak ada client-nya (di negara tsb) yang baca kompas ?[spoiler]# IIX BANNER, I am sure not required much, remove them if not required [jz]
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache[/spoiler]
++++++++++++++++++++++++++++++++++++++++++++++++++++++++

di pakistan ada kompas pakistan kali .....


User avatar
Bandi_Shippuden
Posts: 62
Joined: 04 May 2012, 12:06
Location: Pekanbaru, Indonesia
Contact:

Post by Bandi_Shippuden » 06 Oct 2013, 05:52

Menurut mas g mana tu ...
saya coba di virtual lumayan se HIT youtube nya ...

terus yg masalah resolusi berubah2 di tempat saya udah gak ada ,,,.
ini masih testing ... aq make squid yang dari tempat sampah yg udah suport tproxy juga ...


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 06 Oct 2013, 14:39

Kalau sudah jalan bagus, ya diteruskan saja mas :)


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 07 Oct 2013, 09:34

bandi16 wrote:Menurut mas g mana tu ...
saya coba di virtual lumayan se HIT youtube nya ...

terus yg masalah resolusi berubah2 di tempat saya udah gak ada ,,,.
ini masih testing ... aq make squid yang dari tempat sampah yg udah suport tproxy juga ...
==========================================================

coba mas .... config-nya sampeyan digelar di sini bair semau orang ikut mencoba dan merasakan ...... siapa tau dengan berbagi kita tambah maju dan sukses ngoprex yutubnya


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 07 Oct 2013, 11:59

Sebagai perbandingan "sebelum dan sesudah youtube ber-ubah2 resolusi"


User avatar
papaeky
Posts: 8
Joined: 26 Mar 2013, 15:10

Post by papaeky » 07 Oct 2013, 17:48

q_p wrote:Masak sih berubah sendiri, tanpa campurtangan user ? Coba pada semua kompi client,
  1. anda setting pada high_performance
  2. gunakan clear_type untuk meng-haluskan tampilan font pada lcd
  3. update browser, flash-player dan java


Bisa saja resolusi video berubah-ubah saat mungkin krn seting quality d playernya auto. Dan youtube mungkin mendeteksi kualitas bandwidth da harware kita saat video di play. CMIIW, maklum ga ketemu referensi yg bs jelasin hal ini.


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 07 Oct 2013, 18:22

Secara pastinya saya juga tidak tahu. Selama ini yang kita bahas hanya pada sisi penikmat youtube (baca client/user). Bagaimana pada sisi uploader ? Apakah di sana ada pengaturan resolusi pada saat mereka meng-upload file video, seperti pengaturan pada sisi client di bawah ini ?
Attachments
resolusi_youtube.JPG
resolusi_yusuf
resolusi_youtube.JPG (6.05 KiB) Viewed 1020 times


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 09 Oct 2013, 22:16

q_p wrote:Secara pastinya saya juga tidak tahu. Selama ini yang kita bahas hanya pada sisi penikmat youtube (baca client/user). Bagaimana pada sisi uploader ? Apakah di sana ada pengaturan resolusi pada saat mereka meng-upload file video, seperti pengaturan pada sisi client di bawah ini ?
+++++++++++++++++++++++++++++++++++++
mari kita pecahkan bersama secara cermat tanpa harus mengkudeta pemikiran kita sendiri


User avatar
anbel
Posts: 263
Joined: 18 Mar 2013, 23:38
Location: Bogor - Jakarta
Contact:

Post by anbel » 10 Oct 2013, 23:38

Malam @ All
Bagi teman2 yang resolusi youtube berubah-rubah coba tambahkan add ons SmartVideo For Youtube pada browser dan atur video quality sesuai keinginan kita.
Mudah-mudahan bisa membantu dan yusuf mudah untuk Hit


User avatar
dja
Posts: 61
Joined: 06 Nov 2012, 23:43

Post by dja » 11 Oct 2013, 17:17

sekalian numpang tanya di threat ini mas q_p.. kl di squid log banyak keluar begini

Code: Select all

aioSync: flushing pending I/O operations
gimana obatnya apa mas?? :grin:


Post Reply

Who is online

Users browsing this forum: No registered users and 22 guests