以下为blog主人的回复： 嗯，多谢提供的信息，偶错了，抱歉：）一年前吧，大约。当时找过这方面的资料，没找到。到网站上一找，还不少，特别是论坛上讨论挺多的。粘贴一个：
Re: 1000 concurrent users
由Martín Langhoff发表于2008年09月1日 星期一 19:56
组 Core Developers组 Moodle Partners组 Particularly helpful Moodlers
Ok, here's a quick guide. It does make me sad that I keep writing these notes and noone around here seems to know how to use google.
So - step one: Use google
I'm going to mention several important things that you must use google to find more info about 微笑
Don't use virtualisation
Get real hardware. Virtualisation is for when you have too much HW, and very little to do so being inefficient is not a problem.
Clearly not the case here 微笑
Spend your $ and time on the DB server
The DB server is where performance happens, or not. You will want several cores, lots of RAM, fast, good networking gear.
My recommendation is to buy only stuff that has high quality open source drivers - some kit has only closed source drivers that work on specific versions of RHEL or SuSE. Say no to that. Newer (faster!) kernels will come out, and you'll be left running an old "certified" and slow kernel.
Use PosgreSQL and Linux in 64 bits, and get hold of a good DBA that understands how to tune PostgreSQL and Linux's disk IO for the database. I've posted quite a bit on thos forum about Pg tuning, and there are some excellent guides out there on the net. Pg 8.3 is a speed monster 微笑
Ask the DBA how many spindles to get for the DB system. A good DBA will want at least 2. 3 is better.
If you don't tune it, you'll probably only get 1/10th to 1/4th of the performance that the HW can deliver.
And! Only use a SAN for the DB if you can get guaranteed, assigned spindles.
Moodledata + sessions
I've done moodledata - with sessions in it - over NFS. Not so hot for high scalability.
But at one large install we saw the Oracle clustering FS for this role. Fantastic! I've posted a few times about it, and about GFS (GlobalFS, open sourced by RH).
If you have a SAN handy, GFS or OracleFS are both GPL'd and on all modern linuxes.
Avoid sessions on the DB, use hashed dirs
Your DB will be busy enough as it is. Sessions have high contention, and an ACID DB will spend a lot of effort in the dance around sessions.
Put the sessions on that GFS/OraFS SAN, on NFS or on memcached. If you put them in the FS, make sure you used hashed dirs (I've posted instructions on this in the past, also config-dist.php has some hints).
I am personally not super happy with memcached because
* I'm old school, and memcahed is a cache, not a storage tool. So it will lose data if restarted. There are some scripts that let it save the data.
* Last time I used it showed high tcp latency
* and the C version would crash if the daemon restarted...
* and the pure PHP version was a tad slow
but those are all minor issues.
Go cheap on the webservers
Work hard ensuring you have very low-latency on your network between the webservers and the DB server and moodledata/sessions storage.
And go cheap on the webservers 微笑 you can just buy more of them.
The most important thing on the webservers is memory. I get lots of RAM but go cheap on CPU and disk.
And understand how memory is used, and tune apache for the memory usage. I've posted a ton about maxclients, maxrequestsperchild and related tunables here. Search for my notes on http keepalive timeouts.
One thing we didn't use to have was good info on how much memory each process was actually using -- the COW-related shared memory info was pretty opaque. But recent linux kernels are exposing what are called 'smaps' in /proc, and there are some nice utilities that aggregate that info. It's a bitof a moving target right now - but it gives excellent data on how much memory each apache/php process has that is shared and unshared.
Also - until recently the 64-bits 'webserver' toolchain was not stable - some precompilers acted up, some PHP extensions had trouble. There are advantages to staying on 32 bits for the webservers - including that your mem usage for PHP is smaller. These advantages are starting to disappear, so perhaps this is old advise.
Do use a good precompiler 微笑
It is a good idea to not get too many webservers. Get 2 or 3, and add more when you need them. The more you delay the buy, the more power your dollar buys!
Good tcp-level load balancer
LVS has been great for me. You'll also want to run a reverse proxy - varnish seems to be all the rage lately. Be careful with bad interactions between the LB, rev proxy and httpd.
If you set the LB to be very sensitive things go to hell on a fast train when you start getting real traffic.
Monitor at the lowest level possible
make sure you have iostat & vmstat installed, and are running sysstat. Learn to read all that gibberish and become a true Zen master.
Not kidding. Running sar is always the best way to see what's wrong.
But will this serve N concurrent users?
That's a silly metric 微笑
I tend to measure distinct users in 5 minute windows. The time-window is the most important factor, a few times I've seen people claim of "over 1000 users!" based on 1000 session files in the sessions dir. The sessions dir has a 4hs session expiry 大眼睛
With a good setup (might take a bit of $ on the HW 微笑 ) and good tuning you can do 500 to 1000 unique users in a 5 minute window.
However, it depends what they are doing! Some modules are very light, some are very hard on the servers. Or it may depend on specific options - a simple quiz is lightweight, but you can cram a ton of hard-to-compute stuff on a mod/quiz page.
So you monitor carefully - even the same # of users will change the load on the servers over time:
* they'll use it more as they learn more about it
* they'll learn about new features and start using them - the new features could be heavier - or lighter!
* new versions of moodle will have performance improvements...
* and new modules and features that haven't been performance-tuned
And one trick for your users
If you think about how http is stateless, and how fast computers are, you quickly realise that you can have a few hundred users requesting something off your server in the same minute but they will rarely be concurrent. Think of it for a second -- as long as all the users are distributed over that minute, the webpages are created and served quickly so from the server's point of view, it's dealing with one requester at a time. Two or three, tops.
Dealing with "one or two simultaneous requests" at a any time, it can still do hundreds of thousands of pages as long as they are distributed over time. That's the magic of the webservers - we serve billions of pages over a stretch of time with an infrastucture that is often modest.
Now, imagine a demo situation, or a big lecture hall with 300 students. The lecturer says "and now, we all click 'Search' ". And at that point, in the space of 200ms, 300 replies come. And pile up. Everything goes a bit to hell -- even if the server does not, it's likely that you'll hit some bottleneck in the middle.
So one trick I teach to tutors and lecturers is a bit of indirection. Instead of "and now we all click bla", they say "read the intro text, and when you are done, click bla". The requests get spread in a period of a couple of minutes, and everyone things "wow, this moodle thing is faaaast!" :-p
|
