Try some of the settings for graphics options. FIrstly try turning the textures from High to medium or low. This game as a funny way of dealing with textures and you will find that the swapping involved will appear to be VERY similar to latency problems, instead of your standard FPS drop. Also you may need to turn down some of the dynamic lighting options.
Set lighting and terrain to low and see if it's still laggy. If so, then it probably is your connection. I also found that using hardware sound caused a lot of lag (Audigy 1).
Comment