Squid搭建代理服务器

Squid搭建代理服务器

使用爬虫经常遭到IP被封,网上免费代理经常也不可用,个人写爬虫购买代理没必要,如果有自己的阿里云服务器,则可以使用Squid搭建代理服务器,自己搭建的代理IP稳定可靠,以下将搭建过程记录如下。

安装环境为Centos6.5

first install squid

1
$ sudo yum install squid

then install httpd-tools

1
$ sudo yum install httpd-tools

then create a file to store passwords in it and change the ownership of it so Squid can access

1
2
$ sudo touch /etc/squid/passwd
$ sudo chown squid /etc/squid/passwd

then set password for jack

1
2
3
4
$ sudo htpasswd -d /etc/squid/passwd jack
New password:
Re-type new password:
Adding password for user jack

test password is ok

1
2
3
$ /usr/lib64/squid/ncsa_auth /etc/squid/passwd
jack yourpassword
OK

then in /etc/squid/squid.conf and add following lines to the top of file. and save it

1
2
3
4
5
6
7
8
auth_param basic program /usr/lib64/squid/ncsa_auth /etc/squid/passwd
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
acl ncsa_users proxy_auth REQUIRED
http_access allow ncsa_users

last restart squid

1
2
$ sudo service squid restart
$ chkconfig squid on

Remember to restart the Squid service after deleting the user line in the file

1
jack:ehHDxD10B5KAu

Anonymizing Traffic

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
forwarded_for off
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Cache-Control allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Content-Language allow all
request_header_access Mime-Version allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access Connection allow all
request_header_access Proxy-Connection allow all
request_header_access User-Agent allow all
request_header_access Cookie allow all
request_header_access All deny all
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#### -----------------------------------------------------------
# 配置文件 /etc/squid/squid.conf 可修改监听端口
http_access deny all 修改成 http_access allow all
# 日志目录 /var/log/squid 实时日志
$ tail -f /var/log/squid/ access.log
#### ------------------------------------------------------------
打开浏览器设置代理,如果不成功,关闭防火墙,增加IP端口试试。
$ /etc/init.d/iptables stop
$ iptables -F
$ netstat -punlt
$ vim /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 3128 -j ACCEPT
#### -------------------------------------------------------------
测试 curl -I -s -x http://代理服务IP:3128 www.baidu.com

参考资料:

https://arashmilani.com/post?id=49

http://www.cnblogs.com/apanly/p/5165294.html

http://blog.csdn.net/skylinethj/article/details/43837277

https://github.com/onlyfu/Blog/blob/master/Chrome/%E4%B8%80%E8%B5%B7%E6%9D%A5%E5%81%9Achrome%E6%89%A9%E5%B1%95%E3%80%8A%E6%9C%AC%E5%9C%B0%E5%AD%98%E5%82%A8localStorage%E3%80%8B.md

https://github.com/onlyfu/Blog/blob/master/Other/Vim%EF%BC%8C%E6%9E%81%E7%AE%80%E4%BD%BF%E7%94%A8%E6%95%99%E7%A8%8B%EF%BC%8C%E8%AE%A9%E4%BD%A0%E7%9E%AC%E9%97%B4%E8%84%B1%E7%A6%BB%E9%94%AE%E9%BC%A0%E5%88%87%E6%8D%A2%E7%9A%84%E7%97%9B%E8%8B%A6.md