High load architecture

Architectural solutions are the foundation of any, even high load application building. It is important to understand that the application architecture defines 95% of its success. This also means the ability to cope with stress.

Design principles

Creating a successful, which means a large, web application, you must understand the principles of construction of large systems.

Dynamics

You never know what will happen with application tomorrow. Perhaps your user’s number will increase by 5 times. Perhaps a secondary function will rapidly gain popularity. And it will create a whole new problem. The vaster the system becomes, the less effective a long-term planning will be.

The success of the work on a major project implies not a detailed planning of all aspects. The main effort should be directed at ensuring the flexibility of the system. As it allows to make changes quickly. This is the most important property of any fast-growing system.

Gradual growth

Don’t try to forecast the volume of the audience for the years ahead. The same applies to the application architecture. The basis of successful development is a gradual solution. This applies to both software and hardware.

If you start a new application, it makes no sense to just provide an infrastructure that can support millions of visitors. Use Cloud Hosting solutions for new projects. This will reduce the costs and simplify the server management.

Many cloud hosting services provide a private network. This allows using multiple servers together in a cloud. Thus, you will be able to carry out the first steps in scaling directly in the cloud.

Simple solutions

Simple solutions are extremely difficult to develop. Nevertheless, it is better to spend time and effort to simplify the decision (as for development and users). The flexible system can’t be complicated.

Progressive changes

Working on the big project is like loading a progressive JPEG images. You move not gradually but erratically. You have to constantly refine and remodel different solutions, to switch from one to the other.

95th percentile

95th percentile

Use a 95th percentile rule. You have to spend time only on 95% of your functions. The remaining 5% are usually really special cases that lead to extremely severe complication of the system. For example:

  • Some users in the social network may have tens of thousands of links. If their number is less than 5%, put a limit and do not solve this problem until there are more important issues.
  • Some users upload videos that have a non-standard encoding. Show error instead of wasting time on completion of the conversion.
  • Less than 5% of users use browsers with disabled cookies, Javascript limitations and so on. Disable the site for them and place detailed instructions on updating a browser, instead of adapting the whole site.

Emphasize only important. Remember, you will always have a lot more tasks than time for their solution. Put the right priorities — solve the problems that arise in the absolute majority of users.

Architectural solutions

Scaling of any web application is a gradual process, which includes:

  1. Load analysis.
  2. Determination of the most exposed to high load parts.
  3. Imposition of such areas on the individual nodes and their optimization.
  4. Repeat paragraph 1.

1. Simple application architecture

The new application is usually run on a single server that is already running the web server, the database and the application itself: Simple application architecture

This is reasonable because it saves time and money on the launch. If you are afraid to not withstand the starting load, take a strong server for rent. Only in exceptional situations, when you absolutely sure in a great starting load — go straight to what is described below.

2. Database detachment

Most often, the database is the first node to be subjected to high load. It's clear. Each request from the user usually creates from 10 to 100 queries to the database: Database detachment

Transferring database to a separate server will increase its productivity and reduce its negative impact on the other components (PHP, Nginx, etc.). To connect to MySQL on a separate server use it’s IP address:

mysql_connect('10.10.0.2', 'user', 'pwd');

# 10.10.0.2 is MySQL IP address in the internal network

A pause in the work when moving

Database migration to another server can be a problem for a running application as it will take some time. You can:

  • Use a simple solution — put up a notice about the planned works on the site and make the transfer. Better to do it late at night, when the audience activity is minimal.
  • Use replication to synchronize data from one server to another. In this case, the master is the old server and the new is a slave. After the setting up just change the IP address of the base in the application. Further, read on how to set up MySQL replication on a running server without downtime.

After transferring the MySQL on another server, make sure it has the optimal configuration to get the most out of it.

Database scaling is one of the biggest challenges during the project development. There are a lot of practices — denormalization, replication, sharding and many others. So here are materials on the web apps scaling.

3. Web server detachment

The web server is next in line. Its detachment to a separate node leaves more resources for an application (PHP in this example): Web server detachment

In this case you have to configure the application deployment on the Nginx server and the PGP server. PHP will be a backend server. Nginx will serve static files alone, and PHP server will be busy processing scripts only. Nginx allows to connect to the backend with IP address:

server {
	server_name thehighload.com;

	root /var/www/thehighload;
	index index.php;

	location ~* \.(php)$ {
        fastcgi_pass 10.10.10.1:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
}

# Nginx will serve all static files without requests to the backend

If you are using a file upload, you will need to detach file storage into a single node (about this read below).

4. Several PHP backends

When the load increases, web application begins gradually to work slower. At some point, application implementation will be the reason itself. Then it is necessary to set a couple of PHP backends: Several PHP backends All backends preferred to have the same configuration. Nginx is able to balance the load between them. To do this, you need to create the upstream backend list and use it in the configuration:

upstream backend {
    server 10.10.10.1;
    server 10.10.10.2;
    server 10.10.10.3;
}

server {
	server_name thehighload.com;

	root /var/www/thehighload;
	index index.php;

	location ~* \.(php)$ {
        fastcgi_pass backend;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
}

# Nginx will evenly distribute the load between the specified backends

You can also use weights of the backends if some of them are more powerful than others:

upstream backend {
    server 10.10.10.1 weight=10;
    server 10.10.10.2 weight=2;
    server 10.10.10.3 weight=4;
}

# Out of every 16 requests the first backend processes 10, the second 2, and the third 4

Sessions

Once you start using multiple backends, requests from one user would be transferred to different servers. This will require the usage of a single repository for sessions, for example, Memcache.

5. Caching

Connection caching servers is one of the most simple tasks: Architecture caching

Memcache allows multiple servers to use the standard library:

<?
$m = new Memcache;
$m->addServer('10.5.0.1');
$m->addServer('10.5.0.2');
...
$m->get('user1')

# Connecting memcache to multiple servers at once

Memcache independently distributes the load between the servers. Is uses consistent caching for the purpose. You will need to monitor the evictions and add new equipment in time.

6. Task queues

Task queues let you perform difficult operations asynchronously without slowing the main application. Architecturally, it looks like this: Task queues architecture

The queue server receives tasks from the application. Worker servers process all tasks. Their number should be increased when the average number of jobs in the queue will gradually grow.

7. DNS Balancing

DNS supports round-robin balancing. This allows you to specify multiple IP addresses to hosting web servers (usually referred to as front end): DNS round robin

To use this mechanism, it is necessary to install several identical front ends. Then A records should be specified in the DNS:


....
thehighload.com    IN  A       188.226.228.90
                  IN  A       188.226.228.9x
                  IN  A       188.226.228.9x

# Using multiple IP addresses for a single A record

In this case, DNS will give different IP addresses to different clients. Thus, the balancing between the front end will occur.

8. File storage

Upload and processing of files usually happen in the backend. When you have multiple backends, it is inconvenient:

  • It is necessary to remember where the file was uploaded.
  • Upload and processing files (eg, video or photos) can greatly reduce the performance of the backend.
  • We'll have to use a server with large disks, in what is usually no need for a backend.

The correct solution is to use separate servers for uploading, storing and processing of files: File storage architecture

In practice, this looks like this:

  1. Providing a separate subdomain for the file server.
  2. You need to deploy Nginx and small application that is able to save (and process, if necessary) files on the server.
  3. Scaling is done by adding new servers and subdomains (images 1, images 2, images 3 etc.).

Files upload

Uploading is convenient when is shifted to the client side. Then the form will send requests to a particular server:

<form action="http://images1.thehighload.com/upload.php" method="post" enctype="multipart/form-data">
  <input type="file" name="file">
  <input type="submit" value="Загрузить">
</form>

Domains can be generated randomly from the existing ones:

<form action="http://images<?=mt_rand(1, 10)?>.ruhighload.com/upload.php" method="post" enctype="multipart/form-data">
...

AJAX uploading

To implement AJAX upload, you will need to set the HTTP headers in applications that receive files:

<?
header('Access-Control-Allow-Origin: http://ruhighload.com');
...

# This will send AJAX requests from thegihgload.com domain to the upload domains

Read further about the structure of storage for photos and media data in large projects.

The most important

Remember — not technology but architecture handles high loads. The technologies itself aren’t that important, but right usage is needed. Gradually scale and read more on architectural solutions.

Подпишитесь на Хайлоад с помощью Google аккаунта
или закройте эту хрень