As many have experienced, we've been having issues with lag for quite awhile now, most prominently felt on Wednesday nights. This is largely due to two events that go on during this time, Guild Night and Hand of the Arkati's Reim hunt. For the last several weeks, we've been running tests in the background to track the causes of the lag. We actually thought we found something significant last month, but it turned out to only be adding to the lag, not the cause. Removing that issue was only speeding things up by a few milliseconds at best.
In these tests, we identified 3 issues.
* The room window on how it updates.
* The command queue.
* Spell effects on characters.
The issues aren't the servers, those were all upgraded a few years ago. The issue isn't the GSL layer of the game where the GameMasters work. It's the game engine that is choking. The honest answer is, the game was never built with the level of automation and efficiency that is going on with third-party software. One of our onsite developers is working on resolving two of those issues, the room window (which is largely third-person messaging) and the command queue. GemStone IV is single-threaded, so everything is processed one at a time. When you type in a command, it goes into a queue to execute. Usually this is very quick, so you don't feel it. But as more and more commands get entered, the queue starts to grow to a degree where we see lag. Any time a script inputs a command to check to see if it can do anything else over and over, it's having an overall impact on the game. The room window issue starts to become noticeable when lots of action is going on. Hand of the Arkati's Reim hunt is one of them. Another are places like CCF when groups of over 100 people were all in a room. These things are being worked on though, and we should have some stuff on GemStone IV Test soon (we had some things previously, but we ended up needing to retool a number of things).
The spell messaging is starting to also be an issue as well. We've always had mass spells, and getting them in large groups were always possible. But back in the day it was three mass spells and maybe a few casts of some other things like light/deep blues, a disk, a shot of strength, etc. The spell messaging issues isn't just spell ups either, but it's a major component to it. It's usually what gets things muddied. A Dreavening has everyone walking out with over a dozen spells. That player numbers use to be over 100 because we get messaged whenever the auto-silence effect goes into place. We were told by Dreaven that it's been limited to 60, but it's still not enough to show a noticeable increase on performance. We asked him to reduce that number to 30. If it's not something that we can resolve even by reducing the number, we might have to put group limits into place (like many other games) to see if that helps. This will impact invasions and storylines. We're not trying to go this route, but we have to adapt to how people are playing the game if the tools we have can't support unlimited numbers. And no, there isn't a quick tool for GMs to do to rework this. No flag that would make it easier.
I'll hopefully have some announcements in the next couple of days with some engine updates.