objective ： Let the server support a large number of concurrent access .
notes ： The following ASP/ASP.NET IIS It doesn't work . In addition, be proficient Linux,TCP/IP.
abstract ： This paper mainly introduces the use of single set PC Server to support the concurrent access of millions of users WEB Practical work of server . The purpose is to propose some means to explore the potential of the equipment , Make full use of equipment resources , In order to reduce the cost of hardware investment and maintenance .
With the rapid development of hardware technology , Current single PC
Server performance has been significantly improved , conversely , Hardware costs are falling fast . on the other hand , Most portals , Large communities are under construction WEB service , Mail service and other network services , There is still a tendency to simply expand the number of parallel servers , Storage subsystem to support user requests , Without considering the performance potential of the server itself ! Even if the cost of a single hardware is low enough , The huge amount of basic hardware and its maintenance cost is still a heavy burden for any enterprise !
If you can meet a large number of user requests at the same time , By trying to improve the overall performance of the server , Thus reducing the number of servers , At the same time, it also reduces the cost of equipment maintenance , Can enterprises get more profits ? The answer is yes !
In my opinion , It is necessary to support the concurrent access of millions of users WEB In terms of service , It's really not necessary to have multiple servers , Just a regular one PC The server can do it !
I think there are many at present WEB Business applications are a great waste of hardware investment ! But why do you say that? ?
first , current PC Most servers have multiple independent CPU, And each CPU With multiple hard cores , And many WEB application server , The database server software does not support and optimize the multi-core processor at all
, It's basically a CPU at work , The rest is resting , Didn't give full play CPU The ability of ; Operating system support more CPU, Multicore .
second , Although the server has a high performance network card installed , However, the operating system used does not optimize the network throughput , Unable to support huge networks IO request , Failed to give full play to the advantages of high-performance network card ;
third , Servers generally support 4G Or more memory , And in the x86 The price of memory module on the platform is like Chinese cabbage , Very cheap . Even so cheap and rich memory resources have not been well utilized , most
The application did not consider the memory based caching scheme at all , For example, the most common database access operations in business applications , Usually it is directly connected to the database SQL operation , Its processing performance is not obvious ;
fourth , The application itself is not optimized enough
. Whatever the reason , It is recognized that an un optimized business application has a great impact on the overall performance of the system . We don't talk about advanced optimization technology , Give two common optimization to illustrate , Example 1 , For users who are unique and not always changing in the system ID, Is it possible to consider using a linked list instead of a linked list MAP What about table storage ? The latter is much faster than the former ! Example 2 , For judging whether there is a record in a table , This is usually used SQL sentence ：“select
* from xx_table where xx_id=xx”, Can I change it into “select 1 from xx_table where
xx_id=xx”? One word difference , In the high frequency data query process , The following sentence is much faster than the previous one .
fifth , Generally, there are one or more firewall devices in front of the server cluster , Used for packet filtering and port mapping . If the server's own security is strong , These firewall devices can also be omitted .
For the above reasons , The author and colleagues designed and implemented a single platform PC
The server can support the concurrent access of millions of users WEB The server （ Hardware adoption MS-9188 a main board , Take two Intel(R) Xeon(R) E5410 CPU, each CPU contain 4 Core ,
8G Memory , 320G SATA Hard disk , On board 8 individual Intel 82571EB Gigabit Ethernet , A very common one PC The server ）.
Now we do a brief introduction to our work ：
first , Selection 64 Bit version Linux Symmetric multiprocessing operating system , In order to support more than 4G Memory and multi-core CPU.（ASP/ASP.NET , IIS It doesn't work ）
second , Improve system network IO performance . although Linux The network processing capacity of the operating system itself is relative Windows The system performs better , But to achieve gigabit network throughput is still not enough . This can be done through Smartbits Tools validated . To support millions of user requests , First of all, we should consider improving the network throughput of the server ! We did experiments , It is impossible to achieve the requirement by reducing the kernel protocol stack , The reason is simple , The process of network receiving and sending packets must go through multiple copies of kernel and user space , This is a serious occupation CPU Processing time of . therefore , Our design adopts “ Zero-copy ” technology , utilize DMA Mechanism and memory mapping technology , Realize the fast transmission of network message between kernel space and user space , The process was completely liberalized CPU, So that the data receiving and sending and business processing can be carried out in parallel .
third , Building user space TCP/IP Protocol stack , Establishing connection tracing pool with multi-level fast search capability , To meet the requirements of millions of users concurrent access . This is the key to performance improvement . If you use the original
Linux operating system , Just opening a million file descriptors is a time-consuming operation ! We designed to implement only 1~7 Necessary protocol processing in layer , Protocol processing to remove redundancy in the original kernel , implement
“ streamlined administration ” strategy . The whole protocol process adopts CPU Affinity and other means to achieve more CPU, Multi core parallel processing , More improvement CPU Utilization rate of .
fourth , modify WEB Application server software . The design is open source Apache The server , On the basis of this, we added a pair of “ Zero-copy ” network IO Support for , The number of user requests CPU Parallel processing support , Static page temporary file system storage , Memory database support technology , make the best of “ Time locality principle and space locality principle ”, Significant improvement WEB Performance of server software .
fifth , Improve the security of the server itself . After adopting user mode protocol stack , We realize the overall control of network message , Unpack and pack , And add a variety of DDos Defense of attacks , In addition, the content of the package is reviewed , Its safety filtration standard is much higher than that of simple packet filtration , It completely filters the business content . So the firewall device is redundant , After all, intrusion based on application software vulnerabilities is the main means of current hacker attacks ! And in this respect , Packet filtering firewalls basically do nothing , On the contrary, it is a network bottleneck ! of course , If the firewall is used as a load balancing device, it is another matter .
sixth , Optimization based on specific business application . We optimize the application system mainly from the disk IO, Database access , Business algorithm . A well optimized business system , Its performance will also be significantly improved .
Because we used the above steps , We finally realized the use of one PC
Server support million WEB Requirements of user parallel access . On the whole , The main idea of our design is to maximize the potential of the system , By adopting multi-core concurrency , Network throughput optimization , Business system optimization and other means , Improve the overall performance of the system with multi pronged approach , In order to make good use of the equipment , The concept of using fine equipment , Finally, under the premise of meeting the needs of users , Reduce equipment investment , Purpose of cost saving .