[HOT POOLLING]Apakah Youtube Error Lagi ?

Diskusi tentang Ubuntu Server baik webserver, database server, samba server dan service lainnya serta jaringan menggunakan Sistem Operasi Ubuntu.
User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Re: [HOT POOLLING]Apakah Youtube Error Lagi ?

Post by Rhobbyn » 21 Mar 2013, 01:52

toolooongg...,di tempat saya kalau buka youtube koq yang keluarnya video bokeb.... :(


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 01:57

rhobbyn wrote:toolooongg...,di tempat saya kalau buka youtube koq yang keluarnya video bokeb.... :(

BAhaya ituuu .....


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 21 Mar 2013, 01:59

cah_clacap wrote:
rhobbyn wrote:toolooongg...,di tempat saya kalau buka youtube koq yang keluarnya video bokeb.... :(

BAhaya ituuu .....



iya om...,kasih solusi donk om... :(


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Mar 2013, 02:00

Anda cermati saja postingan di atas :)


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 08:07

sudah ada yg oprex stroreurl belum gan?


User avatar
sekedang
Posts: 8
Joined: 31 Mar 2012, 21:03
Location: aceh
Contact:

Post by sekedang » 21 Mar 2013, 09:28

menunggu pencerahan...


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 21 Mar 2013, 11:29

youtube errorrrrrrrrrrrrrrrrr


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 16:47

Alhamdulillah ..... proxy-ku telah ku install ulang ...... hasilnya lumayan bisa cache video youtube .....
ga tau nanti kalo udah 2 atau 3 video hasilnya akan kaya kemarin apa ga ...


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 16:48

Padahal isinya udah 65% tiap folder cache-nya


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Mar 2013, 17:10

@cah_cilacap
Harusnya, waktu inul dir cache-nya jangan di format.
BTW, boleh lihat storeurl.pl mas ?


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 17:17

@ Mas Pragola : iya betul ..... tadinya memang ngotak atik squid.conf dan storeurl.pl .... tapi lama2 kok bukannya bener malah tambah rusak ....... sudah dikembalikan ke semula malah ga bisa cache satupun file ..... dalam 1 hari jumlah kb -nya ga nambah sama sekali di cache 1 sampai 5 ....... (df -l)

akhirnya ku korbankan semuanya mulai dr nol lg


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 17:17

Ini belum sya coba .... kebetulan warnet lg sepi gara2 youtube error ...


User avatar
Aira
Posts: 70
Joined: 22 Mar 2012, 21:20

Post by Aira » 21 Mar 2013, 18:31

Ikut monitor mudah2an dapet clue & sementara ini masih tetap error.
Mas pragola kenapa ya yang aku ga bisa tercache dalam pecahan apa ada yang salah juga ya...
Mudah2an ga galau neh proxy di otak atik terus seharian...

Mas mallink boleh dunk di gelar sqquidnya biar bisa teratasi lagi masalah youtube nya...


User avatar
phadjart
Posts: 32
Joined: 31 Mar 2012, 21:16
Location: Kota Salatiga
Contact:

Post by phadjart » 21 Mar 2013, 18:52

ini mas.
sama dstdomain nya mas
acl you dstdomain .youtube.com

[spoiler]#!/usr/bin/perl
# $Rev$
# Youtube updates at http://wiki.squid-cache.org/ConfigExamp ... Discussion

$|=1;
while () {
@X = split;
$x = $X[0] . " ";

#facebook
if (m/^http\:\/\/.*(profile|photo).*\.ak\.fbcdn\.net(\/h(profile|photos)-ak-)(snc|ash|prn)[0-9]?(.*)/) {
print $x . "http://facebook.SQUIDINTERNAL" . $2 . "fb" . $5 . "\n";

#speedtest
} elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
print $x . "http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "\n";

#BLOGSPOT
} elsif (m/^http:\/\/[1-4].bp.(blogspot.com.*)/) {
print $x . "http://blog-cdn." . $1 . "\n";

#mediafire
}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
print $x . "http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "\n";

#fileserve
}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
print $x . "http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "\n";

#filesonic
}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
print $x . "http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "\n";

#4shared
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com\/download\/(.*)\/(.*\..*)\?.*/) {
print $x . "http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3\n";

#4shared preview
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
print $x . "http://www.4shared.com.SQUIDINTERNAL/$2\n";

#photos-X.ak.fbcdn.net where X a-z
}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
print $x . "http://photos" . $1 . "/" . $2 . $3 . "\n";

#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
print $x . "http://photos.ak.fbcdn.net/" . $1 ."/". $2 . "\n";

#AVAST
} elsif (m/^http:\/\/download[0-9]{3}.(avast.com.*)/) {
print $x . "http://avast-cdn." . $1 . "\n";

#KAV
} elsif (m/^http:\/\/dnl-[0-9]{2}.(geo.kaspersky.com.*)/) {
print $x . "http://kav-cdn." . $1 . "\n";

#AVG
} elsif (m/^http:\/\/update.avg.com/) {
print $x . "http://avg-cdn." . $1 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . "http://" . $1 . $2 . "\n";

#gstatic and/or wikimapia
} elsif (m/^http:\/\/([a-z])[0-9]?(\.gstatic\.com.*|\.wikimapia\.org.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#Untuk ALL Youtube ( range & non range )
} elsif ($X[1] =~ /(youtube|google).*videoplayback\?/){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@range = m/[&?](range=[^\&\s]*)/;
@begin = m/[&?](begin=[^\&\s]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTER ... redirect\n";

#Google
} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

# compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesy ndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($X[1] =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($X[1] =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($X[1] =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#domain/path/.*/path/filename
} elsif (($X[1] =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($X[1] =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/([^\.]*)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+([0-9]+)?/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#youjizz. We use only domain and filename
} elsif (($X[1] =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url/" . $2 . "." . $3 . "\n";

# spicific extention
# } elsif (m/^http:\/\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv| 3gp|mp(4|3)|exe|msi|zip|on2|mar|swf).*?/) {
# @y = ($1,$2);
# $y[0] =~ s/((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
# print $x . "http://" . $y[0] . "." . $y[1] . "\n";

#generic http://variable.domain.com/path/filename. "ext" or "exte"
#http://cdn1-28.projectplaylist.com
#http://s1sdlod041.bcst.cdn.s1s.yimg.com
} elsif (m/^http:\/\/(.*?)(\.[^\.\-]*?\..*?)\/([^\?\&\=]*)\.([\w\d]{2,4})\??.*$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
print $x . "storeurl://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $X[1] . "\n";
}
}[/spoiler]


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Mar 2013, 19:31

Trim mas, atas sharingnya. Btw, yang seperti itu tadi malam sudah tak coba. hasilnya nihil :confused:


User avatar
phadjart
Posts: 32
Joined: 31 Mar 2012, 21:16
Location: Kota Salatiga
Contact:

Post by phadjart » 21 Mar 2013, 19:54

Pragola_Pati wrote:Trim mas, atas sharingnya. Btw, yang seperti itu tadi malam sudah tak coba. hasilnya nihil :confused:

untuk sekarang lagi ngotak atik melihat access.log nya :D
belajar melihat kejadian serupa


User avatar
RaD3n
Posts: 12
Joined: 19 Mar 2013, 00:51
Contact:

Post by RaD3n » 21 Mar 2013, 20:18

mas mas . om om ...
temen saya pakek virtual proxy
katanya youtube tetep bisa terchace
benerkah sebeb belom cek ke TKP saya


User avatar
budi11
Posts: 97
Joined: 01 Dec 2012, 19:56
Location: Magetan
Contact:

Post by budi11 » 21 Mar 2013, 20:39

RaD3n wrote:mas mas . om om ...
temen saya pakek virtual proxy
katanya youtube tetep bisa terchace
benerkah sebeb belom cek ke TKP saya
minta screeshot access.log nya aja. :grin:


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 21:15

Saya pake squid.conf yg ini GAn .....



#=======================================================================#
## SQUID LUSCA - Youtube Cache-Super Squid Proxy
## http://cah_clacap.blogspot.com
##======================================================================#
## Updated: 22 Maret 2013 | Cah-Clacap
## http://cah_clacap.blogspot.com
#=======================================================================#

http_port 3128 transparent
server_http11 on

pid_filename /var/run/squid.pid
coredump_dir /var/spool/squid/
error_directory /usr/share/squid/errors/English
icon_directory /usr/share/squid/icons
mime_table /usr/share/squid/mime.conf

cache_mem 8 MB
maximum_object_size_in_memory 512 bytes
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA

minimum_object_size 0 KB
maximum_object_size 512 MB
cache_swap_low 97
cache_swap_high 99

cache_dir aufs /cache1 40000 94 256
cache_dir aufs /cache2 40000 94 256
cache_dir aufs /cache3 40000 94 256
cache_dir aufs /cache4 40000 94 256
cache_dir aufs /cache5 40000 94 256

access_log daemon:/var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
store_dir_select_algorithm round-robin
logfile_daemon /usr/lib/squid/logfile-daemon
logfile_rotate 1

acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/24
acl localnet src 192.168.17.0/24
acl localnet src 192.168.100.0/24


acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl purge method PURGE
acl snmppublic snmp_community public
include /usr/local/share/squid/supercache.conf

http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

http_access allow localnet
http_access allow all
http_access deny all

icp_access allow localnet
icp_access deny all
icp_port 0

buffered_logs on

acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast

acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

forwarded_for off
header_access From deny all
header_access Server deny all
header_access Link deny all
header_access Via deny all
header_access X-Forwarded-For deny all
httpd_suppress_version_string on

shutdown_lifetime 10 seconds

snmp_port 3401
snmp_access allow snmppublic all
dns_timeout 1 minutes

#dns_nameservers 8.8.8.8
#dns_testnames 127.0.0.1

#fqdncache_size 4096 # aslinya
fqdncache_size 16384
#ipcache_size 10240 # aslinya
ipcache_size 16384
ipcache_low 97
ipcache_high 99
log_fqdn off
memory_pools off

maximum_single_addr_tries 2
retry_on_error on

icp_hit_stale on

strip_query_terms on

query_icmp on
reload_into_ims on
emulate_httpd_log off
negative_ttl 0 seconds
pipeline_prefetch on
vary_ignore_expire on
half_closed_clients off
high_page_fault_warning 2
nonhierarchical_direct on
prefer_direct off
cache_mgr depkominfo@web.id
cache_effective_user proxy
cache_effective_group proxy
visible_hostname proxy.depkominfo.co.id
unique_hostname depkominfo.co.id
cachemgr_passwd none all
client_db on
max_filedescriptors 8192

# TAG: ZPH
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 21:16

Pakai supercache.conf yg ini ......

________________________________

acl store_rewrite_list urlpath_regex \/(get_video|video\?v|videoplayback\?id|videoplayback.*id)
acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id|watch\?)
acl store_rewrite_list urlpath_regex \.(3gp|mp(3|4)|flv|(m|f)4v|on2|fid|avi|mov|wm(a|v)|(mp(e?g|a|e|1|2))|mk(a|v)|jp(e?g|e|2)|gif|png|tiff?|bmp|tga|svg|ico|swf|exe|ms(i|u|p)|cab|psf|mar|bin|z(ip|[0-9]{2})|r(ar|[0-9]{2})|7z)\?
acl store_rewrite_list_domain url_regex ^http:\/\/(([a-z-]+[0-9-]+)|([0-9-]+[a-z-]+))\.[a-z0-9-]*\.[a-z]{2,4}
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(3gp|mp(3|4)|flv|(m|f)4v|on2|fid|avi|mov|wm(a|v)|(mp(e?g|a|e|1|2))|mk(a|v)|jp(e?g|e|2)|gif|png|tiff?|bmp|tga|svg|ico|swf|exe|ms(i|u|p)|cab|psf|mar|bin|z(ip|[0-9]{2})|r(ar|[0-9]{2})|7z)$
acl store_rewrite_list_domain_CDN url_regex (cbk|mt|khm|mlt|tbn)[0-9]?.google\.co(m|\.uk|\.id)
acl store_rewrite_list_domain_CDN url_regex photos-[a-z].ak.fbcdn.net
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/([a-z])[0-9]?(\.gstatic\.com|\.wikimapia\.org)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$
#acl store_rewrite_list_domain_CDN url_regex ^http:\/\/.*speedtest.*
acl store_rewrite_list_domain_CDN url_regex streamate.doublepimp.com.*\.js\? \.doubleclick\.net.* yieldmanager cpxinteractive quantserve\.com
acl speedtest_allow_url url_regex -i \.speedtest\.net\/ speedtest
acl speedtest_allow_url url_regex ^http:\/\/speedtest\.*
acl speedtest_allow_dom dstdomain .speedtest.net

acl dontrewrite url_regex (get_video|video\?v=|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\? \.asp\? \.aspx\? threadless.*\.jpg\?r=
acl getmethod method GET

storeurl_access deny dontrewrite
storeurl_access deny !getmethod

storeurl_access allow store_rewrite_list_domain_CDN
storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
storeurl_access allow speedtest_allow_url
storeurl_access allow speedtest_allow_dom
storeurl_access deny all
storeurl_rewrite_program /usr/local/share/squid/supercache.pl
storeurl_rewrite_children 4
storeurl_rewrite_concurrency 10

acl DENYCACHE urlpath_regex \.(ini|ui|lst|inf|pak|ver|patch|md5|cfg|lst|list|rsc|log|conf|dbd|db)$
acl DENYCACHE urlpath_regex (notice.html|afs.dat|dat.asp|patchinfo.xml|version.list|iepngfix.htc|updates.txt|patchlist.txt)
acl DENYCACHE urlpath_regex (pointblank.css|login_form.css|form.css)$
acl DENYCACHE urlpath_regex (Loader|gamenotice|sources|captcha|notice|reset)
cache deny DENYCACHE

refresh_pattern \.facebook\.com.*\.(jp(e?g|e|2)|gif|png|tiff?|bmp|swf|mp(4|3)) 43800 99999% 43200 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern \.fbcdn\.net.*\.(jp(e?g|e|2)|gif|png|tiff?|bmp|swf|mp(4|3)) 43800 99999% 43800 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale negative-ttl=0
refresh_pattern ^.*safebrowsing.*google 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth ignore-must-revalidate negative-ttl=10080 store-stale

#ads
refresh_pattern ^.*(streamate.doublepimp.com.*\.js\?|utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 2629742 20% 2629742 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth ignore-must-revalidate store-stale negative-ttl=40320 max-stale=1440

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|watch\?\.flv?|.vid\?) 2629742 99999999% 2629742 override-expire ignore-reload ignore-must-revalidate ignore-no-cache ignore-no-store ignore-private store-stale negative-ttl=0
refresh_pattern (\.swf\?|\.avi\?|\.mov\?|\.wm(a|v)\?|\.3gp\?|\.mp(4|3)\?|\.rm\?|\.ram\?|\.m4v\?|\.on2\?) 43200 999999% 2629742 override-expire ignore-reload ignore-must-revalidate ignore-no-cache ignore-no-store ignore-private store-stale negative-ttl=0
refresh_pattern \.(ico|video-stats) 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod ignore-must-revalidate negative-ttl=10080 store-stale

refresh_pattern (photobucket|pbsrc|flickr|yimg|ytimg|twimg|gravatar)\.com.*\.(jp(e?g|e|2)|gif|png|tiff?|bmp|swf|mp(4|3)) 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern ^http:\/\/images|image|img|pics|openx|thumbs[0-9]\. 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern (zynga|ninjasaga|mafiawars|cityville|farmville|crowdstar|spilcdn|agame|popcap)\.com/.* 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern \.(akamaihd|edgecastcdn|spilcdn|zgncdn|(tw|y|yt)img)\.com.*\.(jp(e?g|e|2)|gif|png|swf|mp(3|4)) 43200 99999% 43200 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-must-revalidate store-stale
refresh_pattern \.gstatic\.com/images\? 43200 99999% 43200 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-must-revalidate store-stale
refresh_pattern (gstatic|diggstatic)\.com/.* 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-private ignore-no-store ignore-must-revalidate store-stale
refresh_pattern ^http://((cbk|mt|khm|mlt|tbn)[0-9]?)\.google\.co(m|\.uk|\.id) 43200 999999% 43200 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-auth ignore-must-revalidate ignore-no-store negative-ttl=10080 store-stale
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 2629742 999999% 2629742 ignore-no-cache override-expire override-lastmod store-stale
refresh_pattern \.speedtest/.* 43200 99999% 432000 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
refresh_pattern galleries\.video(\?|sz) 2629742 999999% 2629742 override-expire ignore-reload ignore-no-cache ignore-must-revalidate ignore-private store-stale
refresh_pattern \.wikimapia\.org\/? 43200 99999% 43200 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-must-revalidate store-stale
refresh_pattern \.(rackcdn|spilcdn|zgncdn)\.com.*\.(jp(e?g|e|2)|gif|png|swf|mp(3|4)) 43200 9999% 43200 override-expire ignore-reload ignore-no-cache store-stale
refresh_pattern code.googlec.com.*(svn|download) 0 50% 1440 reload-into-ims

#sensitive site
refresh_pattern -i \.(sc-|dl-|ex-|mh-|dll|da-|iop) 0 5% 60 reload-into-ims
refresh_pattern -i \.(mst|Xtp)$ 0 50% 1440 reload-into-ims
refresh_pattern -i (main.exe|update.exe|grandchase.exe|FSLauncher.exe|FreeStyle_Setup.exe|grandchase.exe|filelist.zip|autoupgrade.exe)$ 0 50% 1440 reload-into-ims
refresh_pattern -i (UpdaterModifier.exe|FreeStyle.exe|PBLauncher.exe|update.exe|NewLauncher.exe|NewAvalon.exe|hon.exe.zip|cabal.exe)$ 0 50% 1440 reload-into-ims
refresh_pattern -i (PointBlank.exe.zip|HSUpdate.exe.zip|PBConfig.exe.zip) 0 50% 1440 reload-into-ims
refresh_pattern -i (wks_avira-win32-en-pecl.info.gz|wks_avira10-win32-en-pecl.info.gz)$ 0 50% 1440 reload-into-ims
refresh_pattern -i (setup.exe.gz|avscan.exe.gz|avguard.exe.gz|filelist.zip|AvaClient.exe) 0 50% 1440 reload-into-ims
refresh_pattern -i (livescore.com|goal.com|bobet) 0 50% 60 reload-into-ims

#antivirus
refresh_pattern avast.com.*\.vpx 40320 90% 161280 ignore-reload ignore-no-cache ignore-no-store store-stale ignore-must-revalidate reload-into-ims
refresh_pattern (avgate|avira).*\.(idx|gz)$ 1440 90% 1440 ignore-reload ignore-no-cache ignore-no-store store-stale ignore-must-revalidate
refresh_pattern kaspersky.*\.avc$ 2629742 999999% 2629742 ignore-reload store-stale
refresh_pattern kaspersky 1440 50% 161280 ignore-no-cache store-stale

#general
refresh_pattern \.(jp(e?g|e|2)|tiff?|bmp|gif|png) 2629742 999999% 2629742 ignore-no-cache ignore-no-store reload-into-ims override-expire ignore-private ignore-must-revalidate store-stale
refresh_pattern \.(z(ip|[0-9]{2})|r(ar|[0-9]{2})|jar|bz2|gz|tar|rpm|vpu) 2629742 999999% 2629742 override-expire reload-into-ims ignore-no-cache ignore-private ignore-must-revalidate ignore-no-store store-stale
refresh_pattern \.(mp3|wav|og(g|a)|flac|midi?|rm|aac|wma|mka|ape) 2629742 999999% 2629742 override-expire reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-must-revalidate ignore-no-store store-stale
refresh_pattern \.(exe|msi|msp|msu|dmg|bin|xpi|iso|swf|mar|psf|cab) 2629742 999999% 2629742 override-expire reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-must-revalidate ignore-no-store store-stale
refresh_pattern \.(mpeg|ra?m|avi|mp(g|e|4)|mov|divx|asf|wmv|m\dv|rv|vob|asx|ogm|flv|3gp|on2) 2629742 999999% 2629742 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-auth ignore-must-revalidate ignore-no-store negative-ttl=0 store-stale
refresh_pattern -i (cgi-bin) 0 0% 0
refresh_pattern \.(php|jsp|cgi|asx|asp|aspx)\? 0 0% 0
refresh_pattern ^ftp: 40320 20% 40320 override-expire reload-into-ims store-stale
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 50% 40320 store-stale


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 21:18

Dan pakai supercache.pl atau storeurl.pl yg ini

___________________________________________________________

#!/usr/bin/perl
# store rewrite originally writen by chudy_fernandez@yahoo.com bersama adik2nya di STIKI
# modified to satisfy common and dynamic content.
# Suwun loh kang Chudy Fernandez...... kapan2 mampir yo di kampung laut Nusakambangan....
$|=1;
while () {
@X = split;
# $X[1] =~ s/&sig=.*//;
$x = $X[0] . " ";
$_ = $X[1];
$u = $X[1];


#speedtest
if (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
print $x . "http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "\n";

#mediafire
}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
print $x . "http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "\n";

#fileserve
}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
print $x . "http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "\n";

#filesonic
}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
print $x . "http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "\n";

#4shared
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
print $x . "http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3\n";

#4shared preview
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
print $x . "http://www.4shared.com.SQUIDINTERNAL/$2\n";

#photos-X.ak.fbcdn.net where X a-z
}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
print $x . "http://photos" . $1 . "/" . $2 . $3 . "\n";

#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
print $x . "http://photos.ak.fbcdn.net/" . $1 ."/". $2 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . "http://" . $1 . $2 . "\n";

# compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "\n";

# youtube fix
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/videoplayback\?(.*)/) {
$p_str = $2;
$tag = "";
$alg = "";
$id = "";
$range = "";
if ($p_str =~ m/(itag=[0-9]*)/){$tag = "&".$1}
if ($p_str =~ m/(algorithm=[a-z]*\-[a-z]*)/){$alg = "&".$1}
if ($p_str =~ m/(id=[a-zA-Z0-9]*)/){$id = "&".$1}
if ($p_str =~ m/(range=[0-9\-]*)/){$range = "&".$1; $range =~ s/-//; $range =~ s/range=//; }
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $tag . "&" . $alg . "&" . $id . "&" . $range . "\n";

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

# spicific servers starts here....
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";
#...spicific servers end here.

#photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
print $x . "http://photos.ak.fbcdn.net/" . $1 . "\n";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://cdn.yimg.com//" . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

#generic http://variable.domain.com/path/filename. "ext" or "exte" with or withour "? or %"
} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# generic http://variable.domain.com/...
} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
print $x . "http://cdn." . $3 . "." . $4 . "/" . $5 . "\n";

# spicific extention that ends with ?
} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|on2)(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "." . $3 . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $_ . "sucks\n";
}
}


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 21:23

Kendala pertama sy terjadi adalah kalau

________________________________

storeurl_access allow store_rewrite_list_domain_CDN
storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
storeurl_access allow speedtest_allow_url
storeurl_access allow speedtest_allow_dom
storeurl_access deny all
storeurl_rewrite_program /usr/local/share/squid/supercache.pl
storeurl_rewrite_children 4
storeurl_rewrite_concurrency 10
___________________________________________________

kalau setingannya spt ini maka hanya ada 1 video yg tercache ........... sedangkn klo storeurl_rewrite_concurrency 10 angka 10 diganti yg dekat dgn 4 maka ada 2 atau 3 video yng tercache secara acak muncul dikomputer client .......... saya coba jadikan 100 ternyata POP UP yg tambah banyak ....... kalo pakai IDM ketahuan skali klo ada pop up yg keluar ....


User avatar
cah_clacap
Posts: 258
Joined: 19 Mar 2013, 23:07
Location: Cilacap
Contact:

Post by cah_clacap » 21 Mar 2013, 21:24

supaya bisa jalan tanpa tercache paling

storeurl_rewrite_children 4
storeurl_rewrite_concurrency 10

________________________________________

diganti mjd

storeurl_rewrite_children 2
#storeurl_rewrite_concurrency 10


Klo Ada yg salah monggo dikoreksi ....


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Mar 2013, 22:07

@Cah Cilacap
Bentar-bentar, 3 postingan anda ini tujuannya untuk apa sih ? Untuk membatasi jumlah yang ter-cache ?
Ada baiknya dijelaskan dulu :D

Sebagai acuan, di bawah ini basic-nya storeurl_rewrite_children dan storeurl_rewrite_concurrency=
[spoiler]TAG NAME = storeurl_rewrite_children
Description = Specifies the number of redirector process to spawn.If too few Squid will have to wait for them to process a backlog of URLs, slowing it down. If the URL�s are more then it uses the memory and other system resources.
Build Option = Default
Usage = storeurl_rewrite_children number
Default= store_rewrite_children 5
Synopsis = If too few Squid will have to wait for them to process a backlog of URLs, slowing it down. If the URL�s are more then it uses the memory and other system resources. If it runs too many they will use RAM and other system resources.
Arguments = number | Number of location rewrite process to spawn


TAG NAME = storeurl_rewrite_concurrency
Description = The number of requests each store URL redirector can handle in parallel.
Build Option = Default
Usage = storeurl_rewrite_concurrency number
Default = storeurl_rewrite_concurrency 0
Synopsis = The number of requests each store URL rewriter helper can handle in parallel. Defaults to 0 which indicates that the store URL is single threaded.
Arguments = number | Number of requests to be handle[/spoiler]


User avatar
riophone
Posts: 34
Joined: 21 Mar 2013, 21:40
Location: Medan, Indonesia

Post by riophone » 21 Mar 2013, 22:10

Salam Kenal Semuanya buat 0m2 dan Master2
Saya baru gabung di Forum ini

Berikut copyan hasil LOG.


DISK_PARTIAL_HIT/200 18049105 1781747 0 1777664 GET r8---sn-2uuxa3vh-cuil.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&cp=U0hVSFlLUl9OUUNONV9RRVZBOi1YbXBHS1RUY3kz&cpn=iCoscoVbQwdJI7E9&expire=1363904097&factor=1.25&fexp=912301%2C922101%2C919364%2C927810%2C916611%2C913809%2C902000%2C919512%2C913605%2C906938%2C931202%2C900821%2C900823%2C931203%2C931401%2C908529%2C930803%2C920201%2C929602%2C930101%2C930603%2C926403%2C900824&id=o-APICCM5IxT1bdqO2BlfDztOlHev4uiCQNObl6He437iA&ip=36.76.xxx.xxx&ipbits=8&itag=34&keepalive=yes&key=yt1&ms=au&mt=1363878018&mv=m&newshard=yes&range=13-1781759&ratebypass=yes&signature=206DFFA4A2C31EACEA3DD30BF7291CC84B98969C.4FB2A167E0A4C0EE8406E7A02F7DE7B3BBF25225&source=youtube&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&sver=3&upn=kAb2lviiRso (r8---sn-2uuxa3vh-cuil.c.youtube.com) (42508da2e9d08a1d632b0caef647d37a) 0

1363878140 152 192.168.200.31 DISK_HIT/200 18049105 1781760 0 1781760 GET r8---sn-2uuxa3vh-cuil.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&cp=U0hVSFlLUl9OUUNONV9RRVZBOi1YbXBHS1RUY3kz&cpn=iCoscoVbQwdJI7E9&expire=1363904097&factor=1.25&fexp=912301%2C922101%2C919364%2C927810%2C916611%2C913809%2C902000%2C919512%2C913605%2C906938%2C931202%2C900821%2C900823%2C931203%2C931401%2C908529%2C930803%2C920201%2C929602%2C930101%2C930603%2C926403%2C900824&id=o-APICCM5IxT1bdqO2BlfDztOlHev4uiCQNObl6He437iA&ip=36.76.xxx.xxx&ipbits=8&itag=34&keepalive=yes&key=yt1&ms=au&mt=1363878018&mv=m&newshard=yes&range=3563520-5345279&ratebypass=yes&signature=206DFFA4A2C31EACEA3DD30BF7291CC84B98969C.4FB2A167E0A4C0EE8406E7A02F7DE7B3BBF25225&source=youtube&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&sver=3&upn=kAb2lviiRso (r8---sn-2uuxa3vh-cuil.c.youtube.com) (42508da2e9d08a1d632b0caef647d37a) 0

Saya baru coba pakai proxy non Ubuntu yg lisensi trial, hasilnya rata2 yotube tercache semua.


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 21 Mar 2013, 22:49

Selamat datang, selamat bergabung mas :)
.... proxy non Ubuntu yg lisensi tria ...
boleh tahu mas ?


User avatar
riophone
Posts: 34
Joined: 21 Mar 2013, 21:40
Location: Medan, Indonesia

Post by riophone » 21 Mar 2013, 22:55

ThunderCache 7.1 mas
Kalau trial/Free pakai limit


User avatar
juztnewbie
Posts: 3
Joined: 22 Mar 2013, 12:48

Post by juztnewbie » 22 Mar 2013, 12:52

riophone wrote:ThunderCache 7.1 mas
Kalau trial/Free pakai limit
kalau itu bukan linux berbayar mas, setau saya itu sejenis program cache yang di kembang kan oleh orang brazil. memang berbayar.
yang masalah di proxy baik lusca atau squid untuk dynamic content youtube mungkin di storeurl rewriter program (misal storeurl.pl) mesti di ganti source code untuk umpan balik ke youtube, karna keknya code lama itu salah print ID youtube jadi minta video A malah dikasi video B.
CMIIW (lagi oprek)


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 22 Mar 2013, 14:04

juztnewbie wrote: .... yang masalah di proxy baik lusca atau squid untuk dynamic content youtube mungkin di storeurl rewriter program (misal storeurl.pl) mesti di ganti source code untuk umpan balik ke youtube, karna keknya code lama itu salah print ID youtube jadi minta video A malah dikasi video B.
CMIIW (lagi oprek)
:) betul mas


User avatar
phadjart
Posts: 32
Joined: 31 Mar 2012, 21:16
Location: Kota Salatiga
Contact:

Post by phadjart » 22 Mar 2013, 16:30

[spoiler]1363944289.089 35004 192.168.2.2 TCP_MISS/200 1782186 GET http://r2---sn-2uuxa3vh-n0cz.c.youtube. ... tle-factor&
burst=40&
cp=U0hVSFlRVV9MT0NONV9RS1lEOlBqd1lrTkJWRXkz&
cpn=WJffsR8B67co-8N_&
expire=1363967375&
factor=1.25&
fexp=906380%2C902001%2C913564%2C914000%2C916612%2C902000%2C919512%2C913605%2C931202%2C900821%2C900823%2C931203%2C931401%2C913566%2C904476%2C908529%2C919373%2C930803%2C920201%2C929602%2C930101%2C930603%2C900824&
id=o-AJegtSFyGF7hHmK_cnnghq_pEgJScGAfitm2ME6kpghv&
ip=180.246.46.33&
ipbits=8&
itag=34&
keepalive=yes&
key=yt1&
ms=au&
mt=1363944191&
mv=m&
newshard=yes&
range=13-1781759&
ratebypass=yes&
signature=75638BD85E681F9DA06A1EAE1CF10B7389B2F5A0.620FA8385700309B99E56880E23B0A0A1B5BB6D6&
source=youtube&
sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&
sver=3&
upn=8DaPoXNw0cA - DIRECT/118.98.30.205 video/x-flv[/spoiler]
dari log diatas kalo dianalisa untuk helper nya masih bingung.


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 22 Mar 2013, 18:26

phadjart
... dianalisa untuk helper ...
:confused:


User avatar
phadjart
Posts: 32
Joined: 31 Mar 2012, 21:16
Location: Kota Salatiga
Contact:

Post by phadjart » 22 Mar 2013, 18:30

Pragola_Pati wrote:
phadjart
... dianalisa untuk helper ...
:confused:
iya mas, liat access.log nya
buat storeurlnya :)


User avatar
opung
Posts: 3
Joined: 23 Mar 2013, 13:26

Post by opung » 23 Mar 2013, 13:28

sama sy jg error kalo buka youtube yg kebuka cache yg pertama kesimpan


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 23 Mar 2013, 19:42

You tube di tmpat saya udah bisa di cache lagi dan berjalan normal... saya pake script nya bang syaifudin... :D :D


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 23 Mar 2013, 19:46

Baguslah kalau begitu :D
Kalau boleh, warga fui pasti memohon digelar disini mas ...


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 23 Mar 2013, 20:08

Bentar ya mas.... :) :)


User avatar
beling
Posts: 15
Joined: 16 Feb 2013, 20:14
Location: gresik
Contact:

Post by beling » 23 Mar 2013, 21:41

rhobbyn wrote:You tube di tmpat saya udah bisa di cache lagi dan berjalan normal... saya pake script nya bang syaifudin... :D :D
ikutan nunggu juga, izin gelar tikar hehehehehehehe sapa tau scriptnya bisa jalan juga ditempat saya :grin: :grin: :grin:


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 23 Mar 2013, 22:21

build squid dengan "--enable-referer-log"

install paket pendukungnya

# apt-get install libfile-readbackwards-perl

squid config

# referer_log /var/log/squid/referer.log

} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";

.....................................................................................................................



} else {
print $x . $X[1] . "\n";
}
}


dan bagian akhir file tambahkan ini



sub GetID{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline)){
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$ids[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){
$id = $1;
}if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$ids[0].*/){
$id = $1;
}
}
$ref_log->close();
return $id;
}


di tmpat ane pake itu lancar jaya om..
semoga terbantu


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 23 Mar 2013, 22:43

Ok, Terima kasih mas.
Masih ada beberapa pertanyaan yang mnungkin bisa membantu warga fui, sekedar untuk meng-garisbawah-i saja =
1. anda melakukan semua itu dengan install ulang (fressh install) atau cukup dengan meng-compile ulang ?
2. Install tools pendukung

Code: Select all

# apt-get install libfile-readbackwards-per
3. Masukkan option di bawah ini pada saat compile/recompile

Code: Select all

--enable-referer-log
4. Pada squid.conf tambahkan =

Code: Select all

referer_log /var/log/squid/referer.log
5. Pada storeurl.pl =

Code: Select all

} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";
#
#dst
#dst
#dst
#
} else {
print $x . $X[1] . "\n";
}
}
#
#dan bagian akhir file tambahkan ini
#
sub GetID{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline)){
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$ids[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){				$id = $1;
}if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$ids[0].*/){
$id = $1;
}
}
$ref_log->close();
return $id;
}
Kok susunan barisnya berantakan gitu ya ?
Tolong kalau ada yang lebih rapi mas, atau sekalian saja file lengkapnya anda gelar di sini.
6. Support pada squid berapa saja mas ?
Terima kasih banyak mas
You and Syaifuddin, are hero :D


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 23 Mar 2013, 22:58

Pragola_Pati wrote:Ok, Terima kasih mas.
Masih ada beberapa pertanyaan yang mnungkin bisa membantu warga fui, sekedar untuk meng-garisbawah-i saja =
1. anda melakukan semua itu dengan install ulang (fressh install) atau cukup dengan meng-compile ulang ?
2. Install tools pendukung

Code: Select all

# apt-get install libfile-readbackwards-per
3. Masukkan option di bawah ini pada saat compile/recompile

Code: Select all

--enable-referer-log
4. Pada squid.conf tambahkan =

Code: Select all

referer_log /var/log/squid/referer.log
5. Pada storeurl.pl =

Code: Select all

} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";
#
#dst
#dst
#dst
#
} else {
print $x . $X[1] . "\n";
}
}
#
#dan bagian akhir file tambahkan ini
#
sub GetID{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline)){
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$ids[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){				$id = $1;
}if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$ids[0].*/){
$id = $1;
}
}
$ref_log->close();
return $id;
}
Kok susunan barisnya berantakan gitu ya ?
Tolong kalau ada yang lebih rapi mas, atau sekalian saja file lengkapnya anda gelar di sini.
6. Support pada squid berapa saja mas ?
Terima kasih banyak mas
You and Syaifuddin, are hero :D
hahahahaa.. :D :D,maap mas td buru2 mau pulang..,soal nya td mo hujan...
maka nya berantakan.. tolong di atur aja ya mas...

saya cuman compile ulang mas... saya coba di lusca berjalan lancar.... :D :D


User avatar
rimbun
Posts: 5
Joined: 22 Mar 2013, 02:29
Location: medan
Contact:

Post by rimbun » 23 Mar 2013, 23:40

referer_log /var/log/squid/referer.log
susunannya dimana ya bang


User avatar
toni88indo
Posts: 3
Joined: 29 Apr 2012, 15:53

Post by toni88indo » 24 Mar 2013, 00:52

makasih mas akhirnya hit juga tu youtube.
sekarang lagi test biar lebih yakin,,,
:linux:
"2 jempol di atas untuk menunjukan tanda tq"


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Mar 2013, 02:12

@toni88indo
1. mas Toni mungkin ada waktu untuk menceritakan success story-nya. Yakinlah itu sangat berguna bagi warga FUI
2. Konfigurasinya apakah sama dengan postingan tepat di atas ini ? Jika iya, tolong warga di sini dibantu merapikan tata-letak script tsb biar enak dibacanya
Trims :)


User avatar
Rhobbyn
Posts: 7
Joined: 31 Jan 2013, 08:27
Location: Padang
Contact:

Post by Rhobbyn » 24 Mar 2013, 02:49

Ada update dari bang syaifudin biar gak loopback.....,tp ntar di bntu rapiin ya... :D :D


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Mar 2013, 02:55

Biar nanti dikerjakan ramai2 :)


User avatar
adith
Posts: 2
Joined: 24 Mar 2013, 02:47
Location: samarinda,indonesia

Post by adith » 24 Mar 2013, 03:17

newbie nyumbang dikit bang
pakai storeurl bang ucok ...testted youtube hit
ini storeurl


#!/usr/bin/perl
# $Rev$
# by chudy_fernandez@yahoo.com
# rev by ucok_karnadi@yahoo.com
# Youtube updates at http://wiki.squid-cache.org/ConfigExamp ... Discussion

# http://www.facebook.com/stressss

$|=1;
while () {
@X = split;
$x = $X[0] . " ";

#facebook
if (m/^http\:\/\/.*(profile|photo).*\.ak\.fbcdn\.net(\/h(profile|photos)-ak-)(snc|ash|prn)[0-9]?(.*)/) {
print $x . "http://facebook.SQUIDINTERNAL" . $2 . "fb" . $5 . "\n";

#Speedtest
} elsif (m/^http\:\/\/.*\/speedtest\/(.*)\?.*/) {
print $x . "http://speedtest.SQUIDINTERNAL/speedtest/" . $1 . "\n";

#reverbnation
} elsif (m/^http:\/\/[a-z0-9]{4}\.reverbnation\.com\/.*\/([0-9]*).*/) {
print $x . "http://reverbnation.com.SQUIDINTERNAL/" . "$1" . "\n";

#BLOGSPOT
} elsif (m/^http:\/\/[1-4].bp.(blogspot.com.*)/) {
print $x . "http://blog-cdn." . $1 . "\n";

#ytimg
} elsif (m/^http:\/\/i[1-4]\.ytimg\.com(.*)/) {
print $x . "http://cdn.ytimg.com" . $1 . "\n";

#AVAST
} elsif (m/^http:\/\/download[0-9]{3}.(avast.com.*)/) {
print $x . "http://avast-cdn." . $1 . "\n";

#KAV
} elsif (m/^http:\/\/dnl-[0-9]{2}.(geo.kaspersky.com.*)/) {
print $x . "http://kav-cdn." . $1 . "\n";

#AVG
} elsif (m/^http:\/\/update.avg.com/) {
print $x . "http://avg-cdn." . $1 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . "http://" . $1 . $2 . "\n";

#gstatic and/or wikimapia
} elsif (m/^http:\/\/([a-z])[0-9]?(\.gstatic\.com.*|\.wikimapia\.org.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(khm|mt)[0-9]?(.google.com.*)/) {
print $x . "http://" . $1 . $2 . "\n";

# for ALL Youtube ( range & non range )
# first you need do this
# build your squid with "--enable-referer-log"
# install package dependencies "apt-get install libfile-readbackwards-perl"
# add this "referer_log /var/log/squid/referer.log" to your squid config

} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
@itag = m/[&?](itag=[0-9]*)/;
@CPN = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
@IDS = m/[&?]id\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($CPN[0], $IDS[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";

#Google
} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($X[1] =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($X[1] =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($X[1] =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#domain/path/.*/path/filename
} elsif (($X[1] =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($X[1] =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/([^\.]*)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+([0-9]+)?/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#youjizz. We use only domain and filename
} elsif (($X[1] =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url/" . $2 . "." . $3 . "\n";

# spicific extention
# } elsif (m/^http:\/\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf).*?/) {
# @y = ($1,$2);
# $y[0] =~ s/((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
# print $x . "http://" . $y[0] . "." . $y[1] . "\n";

#generic http://variable.domain.com/path/filename. "ext" or "exte"
#http://cdn1-28.projectplaylist.com
#http://s1sdlod041.bcst.cdn.s1s.yimg.com
} elsif (m/^http:\/\/(.*?)(\.[^\.\-]*?\..*?)\/([^\?\&\=]*)\.([\w\d]{2,4})\??.*$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
print $x . "storeurl://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $X[1] . "\n";
}
}


sub GetID
{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline))
{
if ($line =~ m/.*youtube.*\/watch\?.*v=([a-zA-Z0-9\-\_]*).*\s.*id=$IDS[0].*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$CPN[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$CPN[0].*/){
$id = $1;
last;
}
}
$ref_log->close();
return $id;
}


User avatar
anbel
Posts: 263
Joined: 18 Mar 2013, 23:38
Location: Bogor - Jakarta
Contact:

Post by anbel » 24 Mar 2013, 09:25

Saya malah gak jalan ya, ketiga saya
# apt-get install libfile-readbackwards-perl

ada comment E: unlocated

tolong mastah dibantu


User avatar
anbel
Posts: 263
Joined: 18 Mar 2013, 23:38
Location: Bogor - Jakarta
Contact:

Post by anbel » 24 Mar 2013, 10:25

Terima kasih mas robbin & mas adit yg sudah berbagi,
dan terima kasih dan salam hormat buat mas chudy & mas ucok.
Semoga semua dimurahkan rejekinya, Amiiinnnnnnnn,


Akhirnya Youtube bisa dicache dengan sempurna.


User avatar
aadanwde
Posts: 1
Joined: 23 Mar 2013, 08:29
Location: Bogor, Indonesia

Post by aadanwde » 24 Mar 2013, 10:36

Mas.. kalo mau download video dr youtube gimana ya? udah keliling pake cara downverter, cara di folder /tmp youtube-dl -o tetep ga bisa.. bagaimana ya?


User avatar
fida
Posts: 30
Joined: 08 Mar 2013, 03:05

Post by fida » 24 Mar 2013, 10:52

alhamdulillah akhirnya youtube ditempat saya bisa hit lagi setelah saya ikuti cara mas Rhobbyn sudah 4 video yotube saya coba dan semua hit..
terima kasih untuk semua member fui

kalo boleh sekalian saya mau tanya,permasalahan proxy saya ini sudah lama sebelum yotube error,yaitu jika proxy baru dihidupkan lalu saya coba ping dari winbox/pc billing bisa segera terhubung tapi kalo saya coba koneksikan pake putty/winscp lama bgt koneknya kira2 5menit baru konek,jadi kalo proxy reboot setingan netwatch & NAT di mikrotik saya disable dulu.sebab kalo tetep enable lalu proxy rebbot NATnya juga cepat aktif,tapi koneksi belum bisa.mohon penjelasannya para suhu fui...terimakasih sebelumnya..

untuk mas anbel
error waktu # apt-get install libfile-readbackwards-perl
ada comment E: unlocated
coba di # apt-get update dulu baru # apt-get install libfile-readbackwards-perl


User avatar
anbel
Posts: 263
Joined: 18 Mar 2013, 23:38
Location: Bogor - Jakarta
Contact:

Post by anbel » 24 Mar 2013, 11:23

@fida, terima kasih, sudah bisa tadi lupa gak pakai
sudo apt-get...
dan sekarang alhamdulillah youtube sudah tercache dg sempuran.


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 15:12

kenapa # apt-get install libfile-readbackwards-perl

keluar E: unlocated


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 15:16

maksudnya --enable-referer-log gmna tolong pencerahan


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Mar 2013, 15:35

hidden_net wrote:maksudnya --enable-referer-log gmna tolong pencerahan
Masuk ke dir tempat anda build proxy yang dulu dilakukan, dari situ rebuild proxy anda dengan menambahkan opsi --enable-referer-log


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 15:47

ada scrensot nya ga mas Pragola_Pati(saya masih lom ngerti)oh ya facebooknya apa biar bisa tanya2 yang lain di facebook


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Mar 2013, 16:07

hidden_net wrote:ada scrensot nya ga mas Pragola_Pati(saya masih lom ngerti)oh ya facebooknya apa biar bisa tanya2 yang lain di facebook
Cari dengan command locate, contoh kasus begini. Misalnya menggunakan squid-2.7.STABLE9 tapi kita lupa = dulu waktu build/compile letaknya di direktori mana ?

Code: Select all

# locate squid-2.7.STABLE9
ouputnya kira2 begini =

Code: Select all

/home/share_2/arsip/squid/squid-2.7.STABLE9/tools/cossdump.c
/home/share_2/arsip/squid/squid-2.7.STABLE9/tools/squidclient.c
/home/share_2/arsip/squid/squid-2.7.STABLE9/tools/.deps/cachemgr__CGIEXT_-cachemgr.Po
/home/share_2/arsip/squid/squid-2.7.STABLE9/tools/.deps/cossdump.Po
/home/share_2/arsip/squid/squid-2.7.STABLE9/tools/.deps/squidclient.Po
Ketahuan, lokasinya berada di direktori /home/share_2/arsip/squid/squid-2.7.STABLE9. Masuk ke direktori tersebut dan lakukan rebuild/re-compile.


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 16:25

oke sip


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 16:52

ko malah ga ga tersimpan youtubenya ini storurl nya mohon di koreksi


#!/usr/bin/perl
# $Rev$
# by chudy_fernandez@yahoo.com
# rev by ucok_karnadi@yahoo.com
# rev date 02/04/2012
# visit my facebook account http://www.facebook.com/stressss

$|=1;
while () {
@X = split;
# $X[1] =~ s/&sig=.*//;
$x = $X[0] . " ";
$_ = $X[1];
$u = $X[1];

#photos-X.ak.fbcdn.net where X a-z
if (m/^http:\/\/photos-[a-z]?(.ak.fbcdn.net.*)/) {
print $x . "http://photos" . $1 . "\n";

#speedtest
} elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
print $x . "http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "\n";

# photo
} elsif (m/^http:\/\/photos-ash[1-5]?(.fbcdn.net.*)/) {
print $x . "http://ash.photos" . $1 . "\n";

#photos.ak.fbcdn.net where X i(0-9)
#a5.sphotos.ak.fbcdn.ne
} elsif (m/^http:\/\/[a-z][0-9]?.(sphotos.ak.fbcdn.net.*)/) {
print $x . "http://cdn.photos" . $1 . "\n";

#BLOGSPOT
} elsif (m/^http:\/\/[1-4].bp.(blogspot.com.*)/) {
print $x . "http://blog-cdn." . $1 . "\n";

#AVAST
} elsif (m/^http:\/\/download[0-9]{3}.(avast.com.*)/) {
print $x . "http://avast-cdn." . $1 . "\n";

#KAV
} elsif (m/^http:\/\/dnl-[0-9]{2}.(geo.kaspersky.com.*)/) {
print $x . "http://kav-cdn." . $1 . "\n";

#AVG
} elsif (m/^http:\/\/update.avg.com/) {
print $x . "http://avg-cdn." . $1 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . "http://" . $1 . $2 . "\n";

#gstatic and/or wikimapia
} elsif (m/^http:\/\/([a-z])[0-9]?(\.gstatic\.com.*|\.wikimapia\.org.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(khm|mt)[0-9]?(.google.com.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#ini untuk cache partial
} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";

#Google
} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#domain/path/.*/path/filename
} elsif (($u =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://cdn.yimg.com//" . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/([^\.]*)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+([0-9]+)?/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

# spicific extention
# } elsif (m/^http:\/\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf).*?/) {
# @y = ($1,$2);
# $y[0] =~ s/((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
# print $x . "http://" . $y[0] . "." . $y[1] . "\n";

#generic http://variable.domain.com/path/filename. "ext" or "exte"
#http://cdn1-28.projectplaylist.com
#http://s1sdlod041.bcst.cdn.s1s.yimg.com
} elsif (m/^http:\/\/(.*?)(\.[^\.\-]*?\..*?)\/([^\?\&\=]*)\.([\w\d]{2,4})\??.*$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
print $x . "storeurl://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $_ . "\n";
}
}

sub GetID
{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline))
{
if ($line =~ m/.*youtube.*\/watch\?.*v=([a-zA-Z0-9\-\_]*).*\s.*id=$IDS[0].*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$CPN[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$CPN[0].*/){
$id = $1;
last;
}
}
$ref_log->close();
return $id;
}


User avatar
q_p
Posts: 3109
Joined: 14 Oct 2012, 13:01
Contact:

Post by q_p » 24 Mar 2013, 17:01

[spoiler]
hidden_net wrote:ko malah ga ga tersimpan youtubenya ini storurl nya mohon di koreksi


#!/usr/bin/perl
# $Rev$
# by chudy_fernandez@yahoo.com
# rev by ucok_karnadi@yahoo.com
# rev date 02/04/2012
# visit my facebook account http://www.facebook.com/stressss

$|=1;
while () {
@X = split;
# $X[1] =~ s/&sig=.*//;
$x = $X[0] . " ";
$_ = $X[1];
$u = $X[1];

#photos-X.ak.fbcdn.net where X a-z
if (m/^http:\/\/photos-[a-z]?(.ak.fbcdn.net.*)/) {
print $x . "http://photos" . $1 . "\n";

#speedtest
} elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
print $x . "http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "\n";

# photo
} elsif (m/^http:\/\/photos-ash[1-5]?(.fbcdn.net.*)/) {
print $x . "http://ash.photos" . $1 . "\n";

#photos.ak.fbcdn.net where X i(0-9)
#a5.sphotos.ak.fbcdn.ne
} elsif (m/^http:\/\/[a-z][0-9]?.(sphotos.ak.fbcdn.net.*)/) {
print $x . "http://cdn.photos" . $1 . "\n";

#BLOGSPOT
} elsif (m/^http:\/\/[1-4].bp.(blogspot.com.*)/) {
print $x . "http://blog-cdn." . $1 . "\n";

#AVAST
} elsif (m/^http:\/\/download[0-9]{3}.(avast.com.*)/) {
print $x . "http://avast-cdn." . $1 . "\n";

#KAV
} elsif (m/^http:\/\/dnl-[0-9]{2}.(geo.kaspersky.com.*)/) {
print $x . "http://kav-cdn." . $1 . "\n";

#AVG
} elsif (m/^http:\/\/update.avg.com/) {
print $x . "http://avg-cdn." . $1 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . "http://" . $1 . $2 . "\n";

#gstatic and/or wikimapia
} elsif (m/^http:\/\/([a-z])[0-9]?(\.gstatic\.com.*|\.wikimapia\.org.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#maps.google.com
} elsif (m/^http:\/\/(khm|mt)[0-9]?(.google.com.*)/) {
print $x . "http://" . $1 . $2 . "\n";

#ini untuk cache partial
} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";

#Google
} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . "http://www.google-analytics.com/__utm.gif\n";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . "http://" . $1 . $2 . $y . "\n";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . "http://" . $1 . "\n";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

#domain/path/.*/path/filename
} elsif (($u =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://cdn.yimg.com//" . $3 . "\n";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://cdn.yimg.com/" . $3 . "\n";

#for yimg.com with &sig=
} elsif (m/^http:\/\/([^\.]*)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+([0-9]+)?/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://" . $y[0] . ".yimg.com/" . $y[1] . "\n";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

# spicific extention
# } elsif (m/^http:\/\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf).*?/) {
# @y = ($1,$2);
# $y[0] =~ s/((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
# print $x . "http://" . $y[0] . "." . $y[1] . "\n";

#generic http://variable.domain.com/path/filename. "ext" or "exte"
#http://cdn1-28.projectplaylist.com
#http://s1sdlod041.bcst.cdn.s1s.yimg.com
} elsif (m/^http:\/\/(.*?)(\.[^\.\-]*?\..*?)\/([^\?\&\=]*)\.([\w\d]{2,4})\??.*$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
print $x . "storeurl://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . "http://" . $1 . "/" . $2 . "\n";

} else {
print $x . $_ . "\n";
}
}

sub GetID
{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline))
{
if ($line =~ m/.*youtube.*\/watch\?.*v=([a-zA-Z0-9\-\_]*).*\s.*id=$IDS[0].*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$CPN[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$CPN[0].*/){
$id = $1;
last;
}
}
$ref_log->close();
return $id;
}
[/spoiler]
Apaka yang sudah anda lakukan ? coba anda jelaskan kronologi-nya


User avatar
hidden_net
Posts: 20
Joined: 20 Mar 2013, 00:27

Post by hidden_net » 24 Mar 2013, 17:05

sya mengikuti cara ini

build squid dengan "--enable-referer-log"

install paket pendukungnya

# apt-get install libfile-readbackwards-perl

squid config

# referer_log /var/log/squid/referer.log

} elsif ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
sleep(1) ;
@itag = m/[&?](itag=[0-9]*)/;
@ids = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($ids[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/id=" . $id . "&@itag@range\n";

.....................................................................................................................



} else {
print $x . $X[1] . "\n";
}
}


dan bagian akhir file tambahkan ini



sub GetID{
use File::ReadBackwards;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline)){
if ($line =~ m/.*youtube.*\/ptracking.*cpn=$ids[0].*video\_id=([a-zA-Z0-9\-\_]*).*/){
$id = $1;
}if ($line =~ m/.*youtube.*\/ptracking.*video\_id=([a-zA-Z0-9\-\_]*).*cpn=$ids[0].*/){
$id = $1;
}
}
$ref_log->close();
return $id;
}


Post Reply

Who is online

Users browsing this forum: No registered users and 26 guests