Hajas

[BUG] Hitch warning: XXXX msec frame time (Last Update)

Recommended Posts

Posted (edited)

Hello, since the last update my 3 main mods simply start to lag like hell without any errors, just this message appearing in loop:

Quote

Hitch warning: XXXX msec frame time 

Where XXXX are a msec number.

I used to get this when I put so many bots on the server or maps have too much waypoints, like there's too much processing... lowering the number of bots this disappeared and was working great for over a year.

But now came back without ANY change in the mod or number of bots, so something was changed in the last update that is causing more processing than before and causing that.

Any clues about what was changed to cause that? Could you fix that?

Thank you

Edited by Hajas

Share this post


Link to post
Share on other sites

Found the problem and how to fix!

Problem The server simply didn't auto updated to 19.3, and because that are lagging like hell now.

Fix : Download 19.3 server files and update manually your server. Then will auto update to 19.4 and will work fine again.

 

Share this post


Link to post
Share on other sites

Played a lot today with friends and sadly the lags are still there sometimes, but a lot less than before.

Seams something is taking a lot CPU than the last released version, and with that if your server could support 30 bots without problems now will not handle that number anymore because the new version.

What have you done that is using so much CPU than the other versions?

That's sad that you are killing the amazing bots that you created.

Please fix ASAP.

Cheers

Share this post


Link to post
Share on other sites

We didn’t change anything in code to increase the CPU load.

Have you ever considered that your CPU could have become weaker? On shared hosting (VPS etc.) this is always possible.

Share this post


Link to post
Share on other sites

I don't think is that, it's the same computer in my LAN, this PC it's only used for this and not changed for many years and running great.

It's a Intel i5-4250U with 8gb, CPU usage stay between 40-60% load and RAM doesn't reach even to 50%. When the lags happens the machine stays the same, I believe it's something internal COD4X only.

for exemple, I don't touch the code of Star Wars for around 2 years, and this one is also giving this same error.

no errors or lags at Insurgency or world at war, all with bots and heavier than COD4X.

already turned off the Insurgency server, which is the heavier server on this PC, no change.

Turned off all servers in this PC letting just one with COD4X, so would be fixed if was the case right? But doesn't matter, the lags will occurr in larger maps that was working fine before. Check the screenshot proving that, it's just my Frontlines server in a large map. 

After SEVERAL tests I did set a limit of 800 waypoints to be able to handle 30 bots without lags, and was working fine for many time. But now is not handling that amount anymore. I need to re-do all the tests to find the new limits and sadly lower the number of bots/waypoints.

One of the changes in COD4X is that if we test if a bot can see (bullettrace, playerphysicstraces, etc) something, if the destiny origin is INSIDE a solid (player or object), doesn't matter if have several walls between them, now this return as true with the final origin like doesn't have nothing between them, when before that used to return false as supposed to, first hit origin in between (like walls). Was working fine since the first bot functions released in COD4X.

What I did to fix that was create a new function like I did in PezBot to search to the closest waypoint and move there instead, then the bots goes first to the waypoint and only after that he goes direct to the objective. This avoid the bots going crazy.

But maybe that change that was the start of the problem. I'm just guessing here.

Dunno also if the Client might causing that, because I saw that my servers simply wasn't updated as were beeing auto-updated before (dunno why not), so seams the lags started without the servers were updated, but all the clients were with the latest versions.

When I updated the servers, the lags are a lot less now, mainly only with larger maps that have close to 800 waypoints. It's a lot better than without the update, but not as good as was some time ago.

Because that, maybe is a problem with the clients and not the server, since the lags started without change the server.

Another weird thing which I found out lately:

https://cod4x.me/index.php?/forums/topic/5805-platform-strings-are-now-ignored/

Hope you find out what is causing these lags, or just can improving the performance to avoid that.

Cheers

cod4x_lags.PNG

Share this post


Link to post
Share on other sites

Lag started without you making any change to the server? xD

Then we have lowered your servers performance magically? I doubt that, even with client updates.

Share this post


Link to post
Share on other sites

@Fraggy Yes, I don't touched the Star Wars server for over a year. And of course didn't lowered the performance of my servers or PC, but the COD4X servers only.

Testing yesterday the Star Wars in my other instance I started with 20 bots then no errors, 30 no errors, 40 no errors... what?

Not a single "Hitch warning" until the game crash with maximum script variables. (which would be amazing if you could expand this!)

So what's the difference? There's only ONE, the sv_authtoken is not set.

Comeback to my server, commented the authtoken, put the biggest map that we have Dunes (http://bc.hajas.org/cod/mapdb/mp_dunes.jpg) the SAME I posted the errors in the screenshot in the previous message, and guess what?

GONE! Not a single "Hitch warning", no lags, simply great as used to be.

Dunno if is the Client or the Server or both, or maybe any of them, just your master server system is causing lags to list servers. maybe the part that ID bots to show them as different than players? Something that make our server be listed on the master server is causing that.

And of course, I'm not alone with that problem:

https://cod4x.me/index.php?/forums/topic/4462-adjusted-frameend-by-x-snaps-lag/

Any server with Bots started to give these random lags.

For now there's 2 options to fix, or remove the bots or remove the authtoken.

Cheers

 

Edited by Hajas

Share this post


Link to post
Share on other sites
29 minutes ago, Hajas said:

So what's the difference? There's only ONE, the sv_authtoken is not set.

 

29 minutes ago, Hajas said:

in my other instance

didn't you mention that it's not the same instance? maybe i'm misunderstanding something. if you identified sv_authoken as causing hitches, then the same machine with same setup should be used for a comparison with and without sv_authtoken.

29 minutes ago, Hajas said:

why do you think that's the same issue?

Share this post


Link to post
Share on other sites

@dpjI was testing in OTHER INSTANCE when I noticed that.

Then I comeback to my server, REMOVED the authtoken and GONE! No messages, no lags...

It's online now for around 2 hours, no messages yet... you can test if you want, type in console:

Quote

/connect war.hajas.org

I think is the same problem because both started together and are related with bots, and the other messages only occurs when have also players online. Did happened with my servers too, mainly when we have more players online.

But of course, might be another thing, I'll have a better idea after this sunday when we gonna play with the server crowded.

I will update here with the results ASAP.

Share this post


Link to post
Share on other sites

Just a update: without the authtoken the performance is a lot better but is STILL causing lags sadly... a LOT less and most of times playable, but still have lags when didn't had before in the same maps/waypoints. These are the errors:

Hitch warning with just bots online, and Hitch warning + Adjusted frameEnd by x snaps when have also players.

I have some maps with more waypoints than Dunes (which is perfect right now after removed authtoken), some of the maps with more waypoints are still giving this errors. I will need to lower the waypoints of them to make them perfect again.

And again, these waypoints were made a LOOOONG time ago, and were working fine before.

Something is causing that, Client or Server or Both, summing with the Master Server query (authtoken) which if enable is a LOT worst.

Something interntal is consuming a lot more than before, and even just removing the authtoken already make a good difference when shouldn't right?

Cheers

Share this post


Link to post
Share on other sites

I read a lot about waypoints and bots, so why don't you use a plugin to handle the waypoints instead of reducing them all the time?

When the lags are an issue of cod4x then i am with you and agree that they need a fix.

But bots are a heavy thing, especially when you have an eye on the var usage.

moving the waypoint handling to a plugin was one of the best thing i did.

 

lua + modified rotu waypoint (lua) script.

I am currently running a map with 13.000 waypoints (split into areas of 6.000 - anything about 8.000 started to lag) without lags.

  • Like 1

Share this post


Link to post
Share on other sites
On 5/6/2021 at 2:01 AM, Viking said:

I read a lot about waypoints and bots, so why don't you use a plugin to handle the waypoints instead of reducing them all the time?

When the lags are an issue of cod4x then i am with you and agree that they need a fix.

But bots are a heavy thing, especially when you have an eye on the var usage.

moving the waypoint handling to a plugin was one of the best thing i did.

 

lua + modified rotu waypoint (lua) script.

I am currently running a map with 13.000 waypoints (split into areas of 6.000 - anything about 8.000 started to lag) without lags.

might be a last option if the COD4X start to get heavier and heavier... or maybe play something else. I'm really tired to re-code everything again and again. 

Share this post


Link to post
Share on other sites

Just another update, well I just removed the token from all 3 servers and then what?

CPU use decreased from 70-80% to 10-20%, no messages, no lags.

Evidence attached. Definitively there's something very weird about authtoken/list/master server which is causing no need over-processing.

Turn on the authtoken? messages and lags comeback and CPU at 80%

cod4x_lags2.PNG

Share this post


Link to post
Share on other sites

I am running 2 servers (usually have 3 running) with 12 bots in each both with authtoken with no problems.

also running 2 IW4X servers at same time

 

 

resources.png

Share this post


Link to post
Share on other sites

for reference:

here it is with 3 cod4x servers(New Experience / Botwarfare running on each) with 12 bots in each all controlled with B3 bot and 2 IW4x servers with 9 bots in each.

I get the hitch warning but don't get any lag (Hitch warning: 1974 msec frame time)

Can only assume something wrong your end

 

resources.png

Edited by MAD_DAD
extra info

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.