:::: MENU ::::

Free course: The Complete iOS 7 Course – Learn by Building 14 Apps (was $499)

I always want to learn how use build an iOS app, and I have some idea in mind. However, I’m just lazy and procrastinating to do so. Today I found a free course (via Slickdeal) to build an iOS app, and already registered (and you should). It’s free (was $499).

Link: http://bitfountain.io/course/the-complete-ios-7-course-learn-by-building-14-apps/?couponCode=COUPON111677


Markdown is available for self-hosted WordPress through Jetpack

Jay! I’ve just noticed that. It’s funny when searching for Markdown on WordPress return this article:

http://en.blog.wordpress.com/2013/11/19/markdown/

Basically, they mentioned that Markdown is only available for WordPress.com blog, by a simple switch in configuration. I am skeptical, so I always try to use “official” version of plugins. Yes, there are many Markdown plugins out there

http://wordpress.org/plugins/tags/markdown

When I check my Jetpack version, it’s already there! So if you want to write in Markdown style (like Github and StackOverflow), just enable it (why not?).


MySQL bug prevents you from connecting to custom port on MySQL server

It took me a great deal of time and effort to figure out this. In MySQL-client you can specify hostname and port to connect to a different MySQL instance on a different machine and/or different port rather than default localhost instance on your machine. For example, I have 2 MySQL instances running on two different machine, and one of them is behind firewall. Therefore, I need to use SSH tunnel to forward requests to port 3306 of the machine behind firewall.

Things got little complicated when I tried to connect using --port or -P. Since I used the same password for both MySQL server (which I shouldn’t), it took me a while to figure out I still connect to the localhost instance.

The reason is that, when you specify -P only, mysql will switch to socket mode instead of TCP mode. Here is what you need to do:

mysql -P port --protocol TCP

Adding --protocol TCP will force mysql to use TCP connection, thus will connect to the remote instance instead.

Hope that helps!


Dropbox-like synchronization for Linux

One of the requirement for load-balancing servers is server file need to be synchronized. Otherwise, part of your visitors can see your new WordPress post but won’t be able to see the attached photos. rsync can’t do the job properly, because any synchronization tool need to look at the previous state of files in order to determine if new files have been added or any file has been changed or deleted. Fortunately there are several tool:

  • BittorrentSync: This is a fully automatic solution, and as close to Dropbox as possible. You just need to download <code>btsync</code>, generate directory private key and input it into another instance in the other server. Done.
  • Unison: This program run on top of rsync as default (but you can change it), but you need a little trick in order to run it properly. My favorite command is:
    unison -batch -prefer newer -silent -owner -group -times -perms 777 //dir1 //dir2

    The meaning of this command is:

    • -batch: Run in batch mode without asking confirmation
    • -prefer newer: Prefer newer file if conflict
    • -owner: Preserve owner information
    • -group: Preserve group information
    • -times: Preserve time
    • -perms 777: Preserve permission

That’s it. If you know any other tool, let me know in the comment section. Happy sync’ing!


My perfect setup (hint: CloudFlare, DigitalOcean, StartSSL, nginx, apache and private servers)

My situation is a little bit complicated:

  • I have a powerful server completely under firewall (no inbound connection from outside)
  • I want to run several websites (mostly blogs)
  • I want to support SSL

At the beginning, DigitalOcean is the best choice. I will have my own server, host unlimited websites, have full control and DigitalOcean is blazingly fast. I selected the smallest plan with 20G SSD and 512MB RAM. It would be more than enough for my blogs. I installed my own LAMP stack, get my own SSL certificate from StartSSL (you should get your own, too. It’s free!)

Everything is fine until after several week. My server crashed every few hours. There are lots of requests coming for wp-comment-post.php, xmlrpc.php and wp-login.php. Unfortunately I can’t disabled them. Apache’s mod_security and mod_qos couldn’t help much. I have to write a temporary cron script to restart apache2 daemon whenever server load bigger than 20.

Doesn’t improved much. My server still crash. There come nginx. Not work.

Then CloudFlare. The same

Until I decided to use my dedicated server to handle requests. Then it works!, not perfectly but we will be there later.

In short, my configuration is like this:

INTERNET <-> CLOUDFLARE <-> NGINX (DIGITALOCEAN) <-> APACHE (MY DEDICATED SERVER)

There are several technical challenges that need to be solved:

  1. How can I forward requests to my dedicated server (completely under firewall)
  2. How can my end point (apache on my dedicated server) recognize IP from visitors correctly (since there are several layers in between?

The solution for my first challenge is actually very simple: SSH Tunnel. There is one catch: Each website in my dedicated server will have to use its own port. And here is why:

  1. Assume I have 2 websites, example.com and codepie.org. I assigned port 81 to example.com and 82 to codepie.org
  2. Assume my dedicated server has IP 1.1.1.1, my DigitalOcean server has IP 2.2.2.2
  3. In my dedicated server, I fire up a SSH connection with the following parameters
    ssh -g username@2.2.2.2 -R 8081:localhost:81 -R 8082:localhost:82
  4. The command above will create a 2-way tunnel (using -g), an open socket in my DigitalOcean server that will forward all requests to my dedicated server on port 81 and 82
  5. Now you can imagine how I forward request from my DigitalOcean server to my dedicated server: using nginx reverse proxy. My configuration looks like this:
    server {
      listen 80 default_server;
      server_name codepie.org www.codepie.org;
      root /dev/null;
    
      access_log  /var/log/nginx/codepie.org_access.log;
      error_log   /var/log/nginx/codepie.org_error.log;
      
      location / {
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-Ssl     on;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto   $scheme;
        proxy_set_header    X-Frame-Options     SAMEORIGIN;
        
        proxy_pass http://localhost:8082/;
      }
  6. That’s it. All requests to codepie.org will be served by my dedicated server at port 82 (or DigitalOcean server port 8082)

Now come the second challenges. This need to be solved at each layer:

1. CloudFlare. Actually there is nothing can be configured at CloudFlare side. All we need is to add the following lines into /etc/nginx/nginx.conf

http {
    ##
    # CloudFlare
    ##
    set_real_ip_from 199.27.128.0/21;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/12;
    real_ip_header   CF-Connecting-IP;
}

These IPs are nginx cache server of CloudFlare. You can find more information here: https://www.cloudflare.com/ips

2. nginx on DigitalOcean

We need to forward IP information to apache, using this configuration (at host level)

  location / {
    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-Ssl     on;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;
    
    proxy_pass http://localhost:8082/;

3. apache2 on my dedicated server

By using mod_rpaf, apache2 will be able to catch up the correct real IP address

That should be it! If you have any question or suggestion, let me know in the comment section.



Replace tab with space in Vim

I’m a fan of spaces. I hate tabs, actually. It does not look good on vim, git, … you name it. Unfortunately lots of Linux config files are still using tab, like apache2 and so on. I’m like, why don’t they switch to space completely.

The reason is that, under standard screen (80×24), each tab costs 8. After several indentation, you hardly can see anything or almost every line of config will be break into two lines. Doesn’t look good.

Here is a trick to replace tab with space, and ensure vim will use space by default. If you want to use space as default in your system, put it at the end of /etc/vim/vimrc, otherwise at the end of ~/.vimrc

set expandtab
set tabstop=4
set shiftwidth=4

Of course you can replace 4 by any number of your choice. For me 4 is perfect. 2 is too small and 8 is definitely too big.

But what if you want to replace tab with space in existing files? Open vim and type this command:

:retab

Voila! Happy vim-ing.

Source: http://stackoverflow.com/questions/426963/replace-tab-with-spaces-in-vim


Today I Leaned (the hard way): The difference between dict and list and Python

Lookup time for dict() is O(1) and for list() is O(n) no matter what the number of elements is.

Given a list of elements in a file, this piece of code:

from itertools import izip
lists = map(str.strip, open('dict.txt').readlines())
i = iter(lists)
j = range(len(lists))
dicts = dict(izip(i, j))

index = dicts[w]

is n time faster than this piece of code:

dicts = map(str.strip, open('dict.txt').readlines())

index = dicts.index(w)

I learned that the hard way. My dict has 5,000,000 elements, which mean my code will run 5 millions time slower than it should be. As such, I ran my code on 20 machines, each with 64 cores and it took ~10 hours without finish. Changing from list to dict, it took about 900 seconds on a single machine (of course 64 cores)

Next time, when you want to do lookup in Python, use dict(). Happy coding!


CloudFlare + StartSSL is the way to go in mid-October

In my opinion, SSL should be enabled everywhere.

HTTP was designed without security in mind. It was 1989, when Tim Berners-Lee first proposed the “WorldWideWeb” project. At that time, the most important thing is to deliver web content to internet users. Things has changed significantly in the last 10 years. Sniffing plain HTTP content is easier than ever. People care more and more about privacy, especially when Non Such Agency use complicated monitoring infrastructure to spy their own citizens. Google, Facebook, Twitter, banks and other important websites already switched to HTTPS completely.

And few days ago, Google announced that they will take into account whether websites support HTTPS. In short, HTTPS will give you (slightly) advantage over plain HTTP.

SSL certificate has been free to acquire for more than a year by StartSSL. And by mid-October, CloudFlare will also support SSL for free. Therefore, by mid-October, all my self-hosted sites will be completely SSL.

Why not?


Simple tool for load-testing HTTP server

Today I came across a very simple tool for load-testing. It’s called AB and come with Apache HTTP Server. The command line arguments are very simple:

ab -n 10000 -c 10 http://your_web_site/url.html

Where -n is the number of successful requests and -c is the number of concurrence connections.

If you want to have more control over the URL you want to test, and much more, I recommend JMeter (also from Apache)

Source: http://serverfault.com/questions/2107/tools-for-load-testing-http-servers


Pages:1234