Google Search

Google
 

duminică, 21 octombrie 2007

Choke problem and solutions

THE MAIN CAUSE OF BAD CHOKE IS A CLIENTS STEAM INTERNET CONNECTION SPEED BEING SET INCORRECTLY

Please ensure that all clients STEAM Internet Connection Speeds are setup correctly. Choke is quite simply the server wanting to send an update to the client, but cannot.

  • If the server cannot sustain the tickrate, you get choke (You may not actually get choke, but the server will lag very badly)
  • If the server cannot sustain the fps the tickrate requires, you get choke
  • If the server cannot sustain the fps the sv_minupdaterate requires, you get choke
  • If the server cannot sustain the the sv_minupdaterate, you get choke
  • If the server connection cannot sustain the bandwidth required to support the updaterate, you get choke
  • If the server connection cannot sustain the bandwidth required to support the sv_minrate, you get choke
  • If the required bandwidth demanded by the sv_maxupdaterate exceeds the sv_maxrate, you get choke
  • If the clients connection cannot sustain the bandwidth required to support the cl_updaterate you get choke
  • If the clients required bandwidth demanded by the cl_updaterate exceeds the rate, you get choke

Notes regarding Netgraph Updates per second measurements you need to be aware of:

You won't get higher updates than;

a) The servers tickrate

b) The servers sv_maxupdaterate

c) As fast as your server fps allows (Limited by fps_max, hardware and the Kernel Timer Resolution)

d) As fast as your servers sv_maxrate allows

e) As fast as the client/server connection allows

f) As fast as the clients rate allows

g) As fast as the clients cl_updaterate allows

h) As fast as the clients fps allows (Limited by fps_max, hardware, and Refresh Rate)

h) Client FPS controls how fast the client can send updates to the server, this the OUT on the net_graph 3

NB: Other than tickrate, Choke is caused by any of the above list of things not being large enough. This usually occurs because the sv_maxupdaterate or cl_updaterate being higher than the sv_maxrate or rate allows, since the server will not send more data per second than sv_maxrate or rate allows.

Fixing Clients Choke

Besides making sure clients set their STEAM Internet Connection Settings correctly, the only 2 Variables that are going to really help the clients choke problems are RATE and CL_UPDATERATE.

  1. If in doubt about your STEAM Internet Connection Setting, set it 1 higher than what you have.
  2. If you are getting choke and the throughput on the net_graph 3 is lower than what you expect then raise your RATE.
  3. If you still get choke, then make sure you set the CL_UPDATERATE to the servers tickrate and then in steps of 5, lower CL_UPDATERATE until choke disappears or is at least minimised. eg. Start at 100 and try 95, 90, 85, 80 etc. etc.
  4. The blame may not always be with you, the client! Try another server, or another server on another Game Service Provider.

Finally, don't buggerise around trying to fix choke problems if you have loss problems. You are just wasting your time and everybody elses if you ask them to help fix your choke problems if you have LOSS!!! Loss is a network problem

Net_graph 3 Explanation

1. fps is how many frames per second the client is rendering. This is limited by the clients fps_max setting or the refresh rate of the monitors vertical refresh rate if Vertical-Sync is enabled.

2. ping is:

a) netgraph ping is the round trip time for game packets, NOT including any tickrate or updaterate induced calculation delays
b) Scoreboard latency (ping) is one way trip latency (I have to find out in which direction)
c) rcon status command ping, well nobody really knows what this means yet, but I am aiming to find out.

IN is what is being received by you the client, FROM the server.
OUT is what is being sent by you the client, TO the server.

The IN & OUT both have 3 components, starting from left to right:

3. The size of the game packet in bytes being sent and received (Not sure if this includes UDP Segment + IP Packet overhead)

4. The Average amount of KiloBytes Per Second being Sent or Received of GameData + UDP Segment + IP Packet overhead

5. The Average amount of Updates being Sent or Received per Second

If you multiply 3. by 5. and then divide by 1000 you will get a close approximation of the value of 4. which includes rounding errors because 4. and 5. are only averages. So using the numbers we see above, for IN we get (154*102.4/1000)=15.7696 with the value shown in the picture above for net_graph 3 being 15.16. Meh, close enough. :)

The amount of IN Updates received by the client per second (controlled by cl_updaterate) will in most cases equal the servers tickrate, but will NEVER exceed:

  • The Clients cl_updaterate
  • The Servers sv_maxupdaterate
  • The Server/Clients tickrate which are always the same, as the client will always use the same tickrate as the server it connects to

Which ever is the smallest of those 3 numbers will determine the number you see for Updates per second RECEIVED by the client.

If the clients AND/OR servers bandwidth is not sufficient, or is limited by the clients rate or servers sv_maxrate, then the client will NOT see the IN Updates received by the client per second equaling the servers advertised tickrate. This is one example of when the client will see choke.

If the server does not have enough CPU to sustain the servers fps above the servers tickrate, the client will NOT see the IN Updates received by the client per second equaling the servers advertised tickrate. This is another example of when the client will see choke.


The amount of OUT Updates sent by your computer per second (controlled by cl_cmdrate) will NEVER exceed:

  • The Clients cl_cmdrate
  • The Server/Clients Tickrate
  • The Clients Frames Per Second

Which ever is the smallest of those 3 numbers will determine the number you see for Updates per second SENT by the client.

It may look like the OUT Updates sent by your computer per second does exceed the fps, but it reality it does not. It is that the net_graph 3 readings are not always perfectly in sync or there are rounding errors in the calculations, because the the two per second counts 4. and 5. shown in net_graph 3, are only averages. There is also the error in the net_graph 3 that occurs when the Average Updates Received by you per Second magically seem to exceed the servers sv_maxupdaterate, servers tickrate, and the clients cl_updaterate, which were all set to 100 at the time the screenshot was taken above, despite what is shown in the picture above.

6. Loss Are lost packets due to Network problems, either with your computers connection to your ISP, your ISP, or the ISP that is hosting the Server or anywhere in between. If you have loss then you will probably have choke. Do not bother trying to solve Choke problems if you have Loss problems. Resolving loss problems is done by following standard Network Trouble Shooting Procedures. Get a friend to help you or call your ISP, or ask in the Game Server Providers Forum for help. Helping you with network problems is outside the purview of this document, and people who know what they are doing get paid 3 or 4 figure dollar amounts an hour to solve them.

7. Choke Is quite simply the server wanting to send you data but cannot. The reason for this though are not always simple to understand, diagnose or fix.

8. You bring up your net_graph by typing net_graph 3 into console. You may find it helpful to centre the net_graph using the net_graphpos 2 command, and raising it a little so it does not overlay your HUD using the net_graphheight 100 command in console. The net_graphheight command is a function of your screen resolution, so you will need to adjust it accordingly with net_graphheight 100 working well for 1024x768. Increasing the value of net_graphheight makes the net_graph higher and Decreasing the value of net_graphheight lowers the net_graph.

Important Information for both Players and Server Administrators

The Server will not send more data and/or updates than the Client is setup to receive unless the clients violate the server minimums, in which case the servers sv_minrate and sv_minupdaterate will be used by the client.

The Client cannot make the Server send more data and/or updates than the Server is set up to send

You should, after reading the entire article above, now know what it is that controls what the Server & Client can and cannot send & receive, how often, and why.

If you do not, you either did not read what I have written, or some part of my explanation was not clear to you. Suffice it to say all the information you need is in here, even if you do not realise it.

Counter Strike Server and Client rates

During each tick, the server processes incoming user commands, runs a physical simulation step, checks the game rules, and updates all object states. After simulating a tick, the server decides if any client needs a world update and takes a snapshot of the current world state if necessary. A higher tickrate increases the simulation precision, but also requires more CPU power and available bandwidth on both server and client.

HLDS

Half Life Dedicated Server. The program you should be running and trying to optimise if you are here.

FPS

Frames per Second.

Client FPS

Is the number of times per second the game checks for inputs, either from keyboard/mouse and incoming game packets, basically any I/O operation.

Server FPS

Because there are no keyboard or mouse I/O's occurring, it only deals with how often the server checks for game packets.

fps_max

Sets an upper limit on the frames per second the server runs at. Default=300

sv_maxrate

The Maximum amount of data in Bytes per Second the server will send to the client. Conversely, the Maximum amount of data in Bytes per Second the Client can request from the Server. sv_maxrate overrides the clients rate setting if sv_maxrate is less than the clients rate setting. Default=0 Maximum=30000

sv_minrate

The Minimum amount of data in Bytes per Second the server will send to the client. Conversely, the Minimum amount of data in Bytes per Second the Client can request from the Server. sv_minrate overrides the clients rate setting if sv_minrate is greater than the clients rate setting. Default=0

sv_maxupdaterate

The Maximum amount of Update per Second the server will send to the client. Conversely, the Maximum amount of Updates per Second the Client can request from the Server. sv_maxupdaterate overrides the clients cl_updaterate setting if sv_maxupdaterate is less than the clients cl_updaterate setting. Default=60

sv_minupdaterate

The Minimum amount of Update per Second the server will send to the client. Conversely, the Minimum amount of Updates per Second the Client can request from the Server. sv_minupdaterate overrides the clients cl_updaterate setting if sv_minupdaterate is greater than the clients cl_updaterate setting. Default=0

rate

The Maximum amount of Bytes Per Second the Client will request from the server. rate overrides the servers sv_maxrate setting if rate is less than the servers sv_maxrate setting. Default=(Depends Upon Clients STEAM Internet Connection Setting) Maximum=30000

cl_updaterate

The Maximum amount of Updates Per Second the Client will request from the server. cl_updaterate overrides the servers sv_maxupdaterate setting if cl_updaterate is less than the servers sv_maxupdaterate setting. Default=20

cl_cmdrate

The Maximum amount of Updates Per Second the Client will Send to the server. Default=30 Minimum=10 Maximum=100

NB: sv_maxupdaterate and cl_updaterate cannot cause more data to be sent to the client than the sv_maxrate and rate settings allow, or the servers, or most likely the clients actual available bandwidth allows. Choke occurs when either;

The servers sv_maxupdaterate causes the amount of bandwidth to exceed the bandwidth allocated per client by sv_maxrate or the total amount of bandwidth the server has access to,

The clients cl_updaterate causes the amount of bandwidth required by the client to exceed the clients rate setting or the total amount of bandwith the client has access to.

Server Rates

Tickrate is set by adding the -tickrate 100 (for a tickrate 100 server) in the command line start up parameters. Tickrate cannot be changed on the fly via console, HLSW or rcon, it can only be changed in the commandline and the server must be restarted for the change to take effect.

If you want your tickrate changes to have any noticeable benefit you must change a few other server variables as well as change the Windows Kernel Timer Resolution (pingboosting)

To change the Windows Kernel Timer Resolution (pingboost a server) all you need to do is run Windows Media Player. It does not need a file open, it just has be running in the background, if you do not do this, your servers fps will be limited to around 64 frames a second.

The servers fps can be seen by issuing the stats command in console or via HLSW or via the RCON STATS if you are logged in via rcon on the server. The server fps is regulated by the fps_max command (default is 300) which ends up producing around 256 fps in RCON STATS. Don't ask me why, I've asked Valve, but the next step up is to run your fps_max at 600 so you then get 512 fps in RCON STATS. It can be set somewhere between 512 and 600 via console or HLSW and permanently by adding the commandline parameter +fps_max 600, but if you set it at 511 or lower you will see that your FPS according to RCON STATS will still sit at 256 fps even after a map change, so you must set your fps_max higher than what you actually want, to achieve the desired affect. fps_max 512 produces strange results on ping boosted Windows based HLDS, where as fps_max changes on Linux HLDS is affected by many variables, such as, Kernel Version, Kernel Timer, and Hardware so the rules for fps_max that apply to Windows HLDS has minimal relevance to Linus HLDS. DO NOT take my word for it, test it yourself! The reason for running a high server fps, is to ensure that when a server does run a tickrate calculation, that it is using the most up to date information available.

So you have a high tickrate, your server is pingboosted and is running at high fps, none of this is of any use to your clients (the players) if you do not change your servers rates, specifically the sv_maxrate and sv_maxupdaterate variables.

sv_maxrate (default is 0, maximum = 30,000 ). I personally have found that the sv_maxrate 0 setting is detrimental to server performance (purely subjective opinion, but there you go, feel free to ignore it until your clients start complaining about stupid lag and player warping issues that don't correlate to any actual network or cpu usage or over usage issues as the case may be) then set your sv_maxrate to 20000 or if you have player numbers in excess of 20 use sv_maxrate 30,000

sv_maxupdaterate (default 60) setting must be changed to start using all this server generated data more effectively and get the data out to your players who want to run 101/101/20000/10000 cl_cmdrate/cl_updaterate/rate/cl_rate (yes I know cl_rate is defunct but some people can't be told so I humour them and leave it in) settings, thus you need to change your sv_maxupdaterate equal the tickrate. You only have to do this if you run a tickrate higher than 50. eg for tickrate 66 run sv_maxupdaterate 66 or even 100, for tickrate 100 run sv_maxupdaterate 100. If you do not do this, your clients will NEVER see the full benefit of your tickrate changes, and even then, because of server load the clients will not see the full sv_maxupdaterate or tickrate reflected in a net_graph 3. (See below for more information about net_graph 3)

Do not run tickrate higher than 1000 with pingboost on, Valve have admitted that there will be issues if you push the tickrate too high. In fact as of 09January2006 players will have problems on 1000 tickrate servers with pingboost option 3, with doors that won't open and get stuck in spots that will not get them stuck on 66 tickrate servers, e.g. Crouched hard up against boxes on angles.

Make sure you have the bandwidth and CPU to cope with HLDS running with these settings. If you don't have at least a 10Mbps Full Duplex link , you probably do not have the bandwidth to see the full benefit of following the above instructions. This is aimed at people with servers in dedicated data centres with appropriate high speed Internet connections. Most home users will not have the necessary bandwidth or hardware to take full advantage of ALL of these settings. You may though be able to increase your end users overall experience, just by pingboosting your server and increasing the tickrate and fps_max whilst leaving the sv_maxrate and sv_maxupdate rate settings low.

24, 32 & 40 player servers should be run with a tickrate of no more than 66 and sv_maxupdaterate of 100. I've tried higher but you get strange issues for the clients if you do. Well you can if you want, but you need A LOT of CPU dedicated to a single HLDS process.

The fps_max setting of 600 does not appear to hit the CPU as hard as other settings I have mentioned here do, your mileage may vary, but try reducing this back to default of 300 if your clients get strange lag issues and you have tried reducing the other server variables mentioned here. i.e. Change this one last! NB: Your servers fps will not exceed the kernel timer resolution, which varies depending upon what Operating System is being run and how it is setup.

For competition servers, or any server at the 18 player or less mark, then you should be able to use a tickrate of 100 and an sv_maxupdaterate of 100 successfully without any issues, so long as you have the bandwidth and CPU to cope!

For changes to take affect, the settings must be changed in the server.cfg file (except tickrate & fps_max which should be command line variables) and the server restarted, or if done via RCON, a map change must be done.

This was written on the basis that your HLDS is a default HLDS Installation with no Mods/Plugins or Non-Standard anything else, such as sounds, skins, maps etc on the server. Using them (Mods/Plugins) will increase CPU utilisation and thus limit the final result. Obviously you need to monitor these for your particular situation.

Finally, you need to actually play on your server for several hours with all HLDS processes full to see if there are any issues that do not show up by normal performance monitoring tools, to ensure everything is running ok. Subjective in game experience can deviate significantly from Objective Server Statistics, thus you will not know there is a problem unless you are on the server playing at the time it happens.

Client Rates

Clients must have their STEAM Internet Connection Settings setup correctly for their Internet connection.

The clients rate should = the servers sv_maxrate

The clients cl_updaterate should = the servers sv_maxupdaterate which equals the servers tickrate

Thus a server with sv_maxrate 20000 tickrate 100 sv_maupdaterate 100 the clients though run the following settings:

  • rate 20000
  • cl_updaterate 100
  • cl_cmdrate 100
  • cl_interpolate 1
  • cl_interp 0.1
  • cl_smooth 0

These settings will provide the best client experience so long as your server & network can cope with running with a high tickrate and the rates required to take advantage of them.

NB: If your server settings are different to the example just mentioned, your client settings will have to change accordingly. This is just an example, do not think that these rates are optimum for all server settings, they are not, and your optimum client settings will need to change accordingly.

Summary

<> 20 Player servers
-tickrate 66
sv_maxrate 20000
sv_maxupdaterate 66
fps_max 600

Make sure you have the CPU and bandwidth to cope

What you need to look out for is high CPU usage on the server and/or choke on clients that did not get it before you made changes to your servers tickrate and associated settings, and/or fps that running constantly well below the Kernel Timer and/or below the tickrate. Or otherwise, just blatanly obvious crap lag on the server to put it bluntly.

i.e. If you run at 66 tickrate with 50% CPU and 1000 tickrate at 90% CPU then its obvious that 66 tickrate is what you are going to have to run your server at.

If you set your kernel timer to 500Hz or there abouts, and fps_max at 600, but your server is only getting 150-200 fps constantly, then its obvious you need to change the kernel timer and/ore the fps_max to a lower setting.