27 October 2009

lag hoodoo number seven: the server

Early on I mentioned server-side lag as a potential issue. This, as you may recall, is reported on both the Lag Meter and, in much greater detail, the Statistics Bar. The most interesting server numbers on the Statistics Bar as far as I'm concerned are Time Dilation and Sim FPS. Other potentially interesting ones are Agent Updates/Sec and Script Events.

Time Dilation is a measure of how well the region is keeping up. Typically, it should be at 0.99. If it starts to drop below 0.95, the region is lagging, and the lower it drops, the worse the lag. Likewise, Sim FPS is a measure of the fastest rate the sim is able to update the scene to viewers; as with FPS on viewers, if it drops below 15 FPS, you will notice jerkiness, even if your viewer FPS is faster.

So what causes server side lag? The simple answer is complexity: the more there is for the server to deal with, the slower it gets. This complexity can be caused by lots of prims, lots of textures, lots of scripts, or lots of avatars. Starting with the last one first, the server has to keep track of the number of avatars, and the relations between those avatars and everything else in the sim (for our purposes, consider a sim and a region to be the same thing). As the number of avatars in a sim increases, the number of relationships increases exponentially, as all the avatars' viewers have to be updated with information about each avatar. This is why twenty avatars in a sim can seem so much laggier than simply twice ten.

Lots of prims can cause lots of lags, all else being equal. This is true not only of the prims that are a part of the scene, but also the prims that are worn by avatars in the form of hair, shoes, and other attachments. Now the viewer can also be lagged by lots of prims, but the server performs some smarts so as not to send prims that cannot be seen, either because they are completely hidden, or because they are too far away. But the server still has to account for every one of those prims. Likewise, every textures on those prims have to be accounted for, and transferred.

Finally, scripts can cause lag. It's important to know that scripts are handled at a lower priority than the actual rendering of the scene. That means that, if the server is too busy accounting for everything else, it may handle scripts slowly, or not at all. If scripts are running, and there are a lot of them, or one or two that are very processing intensive, they can also add to server side lag.

So, what to do, what to do? Obviously, if you own or have permissions in the region, you can do more than if you are merely a visitor, by reducing both the number of prims and the number of textures. One clever approach is to combine a number of related textures into one, and then using offsets and scaling to use only pieces of that texture on different prims. Likewise, care in the use of scripts can make a big difference. Additionally, laying things out so that not all prims are visible from all locations, while not necessarily helping the server in terms of the number of prims to be managed, will reduce both network and viewer lag.

If you cannot control these aspects of the sim, there are still things you can do. Reducing the number of prims and textures on your avatar can make a difference. This does not mean you have to go naked in a crowded place, but it does mean that careful choice of what you're wearing can make a difference. Avatar Rendering Cost, ARC, is an interesting, though not entirely accurate, means of determining the load your avatar is contributing. This is found from the Advanced menu (activated with Control-Alt-D), as Advanced->Rendering->Info Displays->Avatar Rendering Cost. This will display a number, the ARC, over the head of all avatars in range. Generally, the lower the better, especially in a lag situation. However, notice that the ARC on other avatars, as you move away from them, will drop. This is because at a distance there is less to render on your viewer; but it is still having to be tracked by the sim. Experiment with different hair, or jewelry.

A naked avatar has an ARC of zero, but so does a fully dressed avatar, if she is only wearing appearance (non-prim) clothes and hair. It's the attachment of prims that drives ARC up, and, as we know, prims add complexity to the job the server has to do.

However, it is very important to remember, before you decide to become an ARC enforcers, yelling at others to reduce theirs, that ARC is often really a fairly minor issue compared to the physical presence of the avatar. So the main point in this is to mind your own avatar, and make sure you are not contributing more than necessary to the lag. Also, I recommend you not leave the ARC turned on, as it causes viewer lag for you.

Finally, be a good citizen and don't leave litter around. If you unbox something, don't leave the box lying around. If you pull something out from inventory, or create something for temporary use, remove it when finished.

19 October 2009

lag hoodoo, the sixth visitation

This week we'll talk about competition. The computer has finite resources, and no matter how well you've tuned your viewer, if something else is competing for those resources, it will have an impact.

The biggest potential competitors are other applications. Web browsers, graphics tools, even word processor and spreadsheets can have an impact. Big downloads or streaming media can impact the network performance significantly. If you're seeing a lot of lag, and have tried the things before, start closing applications. If you have lots of tabs open in a browser, close tabs, especially those for high-demand pages such as YouTube.

Vista's (and presumably Windows 7's) Aero interface can have a huge impact on graphic performance. MacOS's interface likewise can compete for graphics card resources. I've found with Vista that disabling the semi-transparent window frames makes a 5 to 10 FPS difference in my viewer performance. Menu-click (that's right click for you right-handers) on Computer to select properties, click 'Advanced system settings', click 'Custom', and then unselect the 'Enable desktopcomposition' box and press OK. Equivalent tuning in MacOS is likely to help.

One other tip is that reducing the area to render improves performance. In other words, if you reduce your viewer from full screen to a smaller window, FPS can increase significantly. This becomes a trade-off between performance and being able to actually see what you're doing sometimes, but is worth experimenting if you are still suffering from lag issues.

Next week, we'll discuss server-side lag, and what can be done about it.

12 October 2009

lag hoodoo episode V: the network strikes back

Moving on from graphics settings, it's time to take a look at what the network information means, and what can be done about it. In the Lag Meter (Help->Lag Meter) the second item displayed is labeled Network. In the Statistics Bar (Ctrl-Shift-1, or View->Statistics Bar), there are three items, 'Bandwidth', 'Packet Loss', and 'Ping Sim'. There are more network details under the 'Advanced' section of the Bar, but we will not go into them in this series.

Bandwidth means the total amount of information than can be transferred in a given period of time, while ping time is a measure of latency, that is, how long it takes to transfer individual packets of information. A high bandwidth connection with with large latency (a satellite internet connection, for instance) will likely feel jerky, even though all the information is being transferred rapidly. Likewise, a low bandwidth connection (such as dial-up, ISDN, or slower DSL) will feel like moving through treacle or molasses, even if the latency is very low.

Network-related items that can be controlled within the viewer are found in the Preferences window (Ctrl-P, or Edit->Preferences...), and include 'Maximum Bandwidth' and 'Disk Cache Size'. The default setting for bandwidth is almost certainly too low, but the correct setting will depend upon the specifics of your ISP (Internet Service Provider), local network, and computer hardware. The quick way to find the appropriate setting is to move the slider to the right until you start to see Packet Loss in the Statistics Bar, and then slide it back to the left until the loss is 0.0%.

Disk Cache Size specifies how much room to use for caching downloaded texture and other information, and should always be set to the maximum value, unless you are very restricted on local disk space. The slider will go to 1000 MB, and should be set there. In fact, there is a way to go even higher through debug settings, but at some point you reach diminishing returns. You should not press the 'Clear Cache' button except to try to resolve a rendering or crashing problem, or you will rob yourself of much of the benefit of caching.

The Second Life Snowglobe viewer has introduced a new texture downloading technique than can, in some cases, greatly improve texture downloading and caching. This improvement has also been included into some other viewers, including Emerald and Kirsten's

Outside of the viewer, there are some additional things that will affect network performance. Everybody knows that cable internet connections are always faster than DSL. Of course, everybody is often wrong, and the truth is much more complex. DSL is often available in a range of speeds, and an upgrade can often provide significantly better performance. DSL also is a switched system, meaning that a neighbour's downloading does not impact your performance as directly as it does with cable. On the other hand, DSL speeds degrade rapidly depending on your distance from the local switching office. Other ISP systems, such as wireless broadband, have their own issues.

On your local network, using a wireless connection can sometimes have a very large impact. If network performance continues to be poor, try a direct wired connection to see if that is the issue. If it is, the solution may be as simple as moving the wireless router a small distance, or to another room closer to where you use the computer, or may require upgrading to a faster wireless specification, or even giving up using it all together.

Ping Sim, or latency, is a function of the distance from the server to the viewer, and is not normally something you can control, short of moving physically closer to the servers. There can be situations caused by bad routing that make this distance needlessly long. Likewise, lots of Packet Loss even with a very low Maximum Bandwidth setting can indicated a hardware issue somewhere in the path from the server to your computer, or a misconfiguration of a router or firewall. Diagnosing general network performance problems is beyond the scope of this article, though it's amazing how often simply replacing a cable can make a difference.

disclaimers

First of all, I have never yet—to the best of my admittedly fallible memory—blogged about anything given to me gratis. That doesn't mean I won't ever do so, or that I think doing so is a great evil. It just hasn't happened.

Second, I believe that disclosing that an item being reviewed was received free of cost is ethically the right thing to do, regardless of any legal requirements. It's simply a matter of basic honesty.

Those two things said, I personally suspect the US FCC's strong recommendation that bloggers make disclosure treads dangerously close on treating the speech of bloggers differently from that of other media. Would such treatment stand up in the US courts? I don't know; as a strong free-speech supporter, I certainly hope it would not.

In any case, any blog reader who assumes that, because the FCC is enforcing such a rule, all bloggers will adhere to it is both naïve, and does not grasp the international nature of the Web. I certainly hope that FCC commissioners and staffers don't fall into those categories, but I do wonder. After all, this is the same government agency that has spent, and continues to spend, vast quantities of money in a tizzy over Janet Jackson's nipple.

06 October 2009

lag hoodoo: la cuarta parte

We'll wrap up our discussion of lag caused by viewer graphic settings in this instalment, taking a look at the 'Hardware Options' button off the main Graphics tab in the Preferences window. In the smaller window that opens with this button contains half a dozen settings. In order, these are:

Filter: this is essentially a way of making textures look more realistic, but it comes at a performance cost. It's probably best left off, unless you have a very powerful system, or are making photographs.

Antialiasing: this smooths the jaggy edges of angles. Again, it comes at a cost, and the higher the setting, the higher the cost.

Gamma: this is an adjustment to image brightness, and is probably not worth worrying with. In my experience, it does not make much if any difference to performance, but adjusting it may make scenes more vibrant in some cases. See Wikipedia for more detailed information. In any case, if you are using Windlight adjustments, I believe they override this setting.

Enable VBO: the Vertex Buffer Object extension to OpenGL is a setting to eliminate the calculation and transfer of redundant information in rendering objects. It's generally best left on, but experiment if you like.

Texture Memory (MB): the size of this is determined by available memory on the graphics card, and helps prevent the constant re-downloading of textures from the server. According to Tateru Nino, the SL 1.22 viewer seldom uses much more than 256 MB. Presumably, this is also true of the other viewers. If you have it set higher, and see problems with crashes, adjust it downwards.

If you are running multiple viewers, I suspect, but cannot prove, that it's best to divide the total cache size of your graphics hardware by the number of viewers you are running, and set the Texture Memory no higher than that value.

Fog Distance Ratio: this affects how quickly things fade out in the distance, but is only used if 'Basic Shaders' on the main graphics tab is disabled. The lower the value, the closer the fog, and presumably the better the performance, though I haven't noticed huge effects.

One last item to discuss, back on the main graphics tab, is combination of 'Run Second Life in a window' and 'Window Size'. In my experience, setting the window size smaller than full screen can improve FPS performance, sometimes significantly. Of course, this becomes a trade-off between performance and being able to actually see anything.

Next time, we'll discuss the network performance settings. Until then, may your rendering be beautiful and your lag be low.

29 September 2009

lag hoodoo part III




Last week, we took an initial look at the Graphics settings in the Preferences dialogue in an effort to improve our frames per second (FPS) performance. This week, I was planning to take a closer look at all the options on that page, but, due to a combination of jury duty, flu in the family (not I, thankfully!), excitement at winning a chariot race, and plain old forgetfulness, I'll just have to make it short, and touch on a couple of things.

If you open the Preferences window, and select the Graphics tab, next to the the 'Quality and Performance' slider mentioned last week, there is a box labeled 'Custom'. If you select this, a score of controls will spring into view. Some of these may be greyed out, depending upon the Quality and Performance setting and the capabilities of your graphics card.

There is also a 'Recommended Settings' button at the bottom of the window, which will reset the settings to a consistent starting point, and unselect the Custom box at the same time. This is seldom the optimum result, but does provide a way to quickly get back to a known state while experimenting with settings.

If, after opening the Custom controls, you move the Quality and Performance slider, you will find that it is actually adjusting each of these controls. Play around a bit, and observe how the settings change, what their visual effect is, and what their impact on FPS is (Ctrl-Shift-1, remember). Then push the Recommended Settings button again, and re-select Custom.

While I have found all of these settings have some effect, some have more than others. Finding the optimum setting is a matter of trading off performance you can live with for visual quality you are happy with, and these values may change depending on the region you are in, and what you are doing. For instance, when I am chariot racing or in a sailing regatta (did I mention I'm the champion of the Minoan Empire in both sports?), I will set every thing to the absolute lowest setting, because I value performance above every thing else. For most purposes, though, I run at High Quality, with several of the other controls tweaked a bit.

I will wrap this up for now by noting that 'Draw Distanc'e is one of the most powerful controls if you are in a region with a large number of prims, avatars, and/or textures. If you are in a very empty area, it will not make as much difference. I have also found that the 'Water Reflections' and 'Atmospheric Shaders' selections under 'Shaders' can have a big effect.

Until next week!

23 September 2009

lag hoodoo part two

As we look at the Lag Meter, under the Help menu on most OpenSim/Second Life viewers, we see three indicators. In this post and the next two, we'll look at what those mean, and how a poor, lagged down user can influence them.

The first one is the Client. This indicator is presented as FPS, or Frames Per Second, and is an indicator of how quickly the local viewer is able to re-render the scene. The faster it renders, the smoother and more immersive the experience will feel. For comparison, standard resolution television renders at 25 or 30 FPS, HDTV ranges from about 24 to 60 FPS, standard 35 mm cinema film runs at 24 FPS, and IMAX runs at either 24 or 48 FPS.

The Lag Meter flags any rate below 10 FPS as red, and below 15 as yellow. My personal observation has been that, below 10 FPS, it starts to feel progressively more and more as if I am trying to walk through hardening cement, and appears more and more as if I am moving through a series of slowly changing pages (which is essentially what is happening). Above 10 FPS, it's generally easier to move, and above 20 FPS or so, it begins to appear much more smooth and lifelike. I have also found that in competitive activities, such as sailing and chariot racing (ask me sometime), if the frame rate drops below 15 FPS, control of my vehicle becomes much more difficult.

So, one might ask, what influences the frame rate? Why doesn't the viewer simply render at 25 or 30 FPS, and be done with it? The answer is both simple and complex, and we'll take more than one week to answer it, but the short answer is, rendering an entire scene can be very difficult, and is impacted by anything that increases complexity. A scene rendered on your computer consists of polygon edges (the wireframe), textures, and lighting. This is true not only of the land and prims, but of all the avatars within the view. The viewer must render all of these things as quickly and efficiently as possible. Having few visible prims makes the job easier, while lots of prims, with lots of textures, makes it harder. Static things make it easier, while moving things--avatars, flexiprims, lots of particles--make it harder.

The rendering is primarily the task of the GPU, the graphics processing unit, and it's supporting electronics on the graphics card. This is an area in which all are definitely not created equal. Take a look at this somewhat outdated http://s3.amazonaws.com/secondlifegrid.net/frameratesbygpu.png (comparison of frame rates by GPU) document. This is specific to older versions of the Second Live viewer, and does not include some of the newer graphics processors, but the relative comparisons still hold true. If anything, newer versions of the viewers are more demanding, and the typical rates will be lower. The ranges are due to several things, including the differences in simple and complex scenes. Also note that ATI GPUs are typically somewhat slower than equivalent nVidia GPUs, even though in a game such as World of Warcraft they may be much closer. Without going into detail, this appears to primarily a difference in drivers between the two GPU makers.

Second to the GPU, the local computer's CPU also handles a lot of the calculations, and the weaker the GPU, the harder the CPU will have to work to compensate. Third, there is memory caching, in both the graphics card and main system RAM. All else being equal, more RAM on both the system and the graphics card is a good thing, as it allows things to be kept available for quick use, instead of having to be retrieved from the server via the network, or from the local hard disk. Finally, there is the network itself. While network is a separate entry on the Lag Meter, it also affects FPS. A processor, whether GPU or CPU, that is not being given things to process as fast as it can handle them simply cannot work at its maximum potential.

So, given that, what are some quick and easy things we can do to increase FPS? There are things that will only affect the local system, and things that will affect not only the local system, but other users as well. We'll start with the former. I will be using the most current Hippo viewer version 0.5.1 set to the English language for consistency as I describe the options; other viewers and other languages may cause some of the settings to vary, or be differently labeled.

The very first thing to do is to open the Preferences window, either Control-P, or from the pull down menu, Edit->Preferences. Select the Network tab, and move the slider for the Disk Cache Size all the way to the right. This maximises the amount of on-disk storage for textures retrieved from the network. Click Apply. Note that this will increase the disk space used locally. You may not see an immediate increase in FPS, but over time, it should help in areas you frequent. Note: if you do not have sufficient space available to increase this cache by 500 MB, then you really should consider either adding more disk space, or removing something from your system.

The next step is to select the Graphics tab. If you have never made adjustments here, you should see a fairly simple dialogue, with a single slider for Quality and Performance. Unless it is already in the leftmost position, try sliding it one step to the left. You should immediately see an increase in FPS on the Lag Meter (Help->Lag Meter) or the Statistics Bar (Control-Shift-1), at the cost of some of the quality of the rendering. This is because the number of details being rendered is being decreased to reduce the rendering time, and thus increase the FPS.

Next time we'll go into more detail on this tab, but for now, if you'd like to experiment more, tick the "Custom" box. This will open up a number of boxes and sliders that you can test to see what effect they have on your system's performance.

One last thing to experiment with is decreasing the size of the viewer window. Tick the box at the top of the graphics tab, labeled 'Run Second Life in a window', if it is not already. Then try choosing different Window Size settings, or simply resize the window frame. You may find that, at smaller window sizes, you see an increase in FPS.

Until next time, cybernauts, may your lag be low!

~*~
Thoria