Rithmic latencies - HW,windows or rithmic? possible workaround

Market data, Trading, Technical support, Features requests, etc
dap
Posts: 19
Joined: Mon Mar 09, 2020 6:38 pm
Has thanked: 27 times
Been thanked: 2 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by dap » Wed May 20, 2020 9:28 am

Tested this morning from 9AM to 11AM CEST time, not much movements, so we can exclude the amount of data received like at cash open

I had 3 delays above 8 seconds (and also more around 4/5 seconds)

Today the delays have all in common 1 logic processor used at 100%, others were pretty unloaded. I put r pro only on "even" core 8,10,12,14. Bookmap from 0 to 7

Attached a 19 seconds delay, warned by bookmap red label and "slow computer" alert from r pro
Also the 8.6 seconds delay

 
Attachments
rith_pari_8.6sec_1cpu.JPG
rith_pari_8.6sec_1cpu.JPG (75.98 KiB) Viewed 4530 times
rith_pari_19sec_1cpu.JPG
rith_pari_19sec_1cpu.JPG (63.94 KiB) Viewed 4534 times

dap
Posts: 19
Joined: Mon Mar 09, 2020 6:38 pm
Has thanked: 27 times
Been thanked: 2 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by dap » Wed May 20, 2020 10:08 am

mpxtreme wrote:
Wed May 20, 2020 4:30 am

I have a nearly identical setup as the original poster and was considering an upgrade.
Ryzen 1700x ( upgrade to 3700x?)

Hi, mpxtreme

I don't think upgrading to 3700x/3800x will change much, the frequency and IPC improvement are not so big, I think you'll have the same problems

https://www.tomshardware.com/reviews/ry ... 214-3.html

Svyatoslav
Site Admin
Posts: 88
Joined: Mon Jun 11, 2018 11:44 am
Has thanked: 2 times
Been thanked: 7 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by Svyatoslav » Fri May 22, 2020 5:22 pm

Hi. We are still investigating it, but unfortunately not much info yet. Could you please try the following test: when the issue happens (CPU pinned + latency), take a screenshot of R Trader, specifically of this area:
Capture.PNG
Capture.PNG (3.24 KiB) Viewed 4472 times
Those icon colors should be different depending on where the bottleneck is, so that would point us in the general direction of where the problem might be.

Also, could you please send the following log files (probably send to me directly as a private message or to our support support@bookmap.com and tell them to forward to Svyatoslav):
- Rithmic log file from Bookmap: "C:\Bookmap\Logs\rithmic-engine-log.000"
- R Trader log file from "C:\Users\<username>\Documents\Rithmic\Rithmic Trader.cur.txt"
- Bookmap log, name similar to "C:\Bookmap\Logs\log_20200505_164543_045-common.txt"
Ideally please also write when exactly you have observed the issue, so we know at which point in files it is expected to be

Regarding the upgrade to 3700x - we didn't figure out the issue yet, so it might be too early to consider that. While it should have better per-tread performance, the difference is most likely not big enough to solve the issue by itself.

UPD: requested logs

Thanks,
Svyatoslav

dap
Posts: 19
Joined: Mon Mar 09, 2020 6:38 pm
Has thanked: 27 times
Been thanked: 2 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by dap » Fri May 22, 2020 7:45 pm

Sorry I didn't follw too much today but here some examples, all 4 icons green. r trader pro on 10,12,14. bookmap from 3 to 9

The screenshots hours are 20:34, 21:30 (UTC+2), but I see in bookmap common log all the spikes detected, I'll send an email, didn't find the attach in private messages

However to me seems an r trader problem, which uses only 1 cpu at 100%. 
But before the bookmap api connection update I never had r pro open, so cannot compare it

You didn't reproduce this behaviour yet? No cpu spikes on your end?  

 
Attachments
rith_lat_ok_8sec.JPG
rith_lat_ok_8sec.JPG (110.15 KiB) Viewed 4452 times
rith_lat_ok_2sec.JPG
rith_lat_ok_2sec.JPG (114.63 KiB) Viewed 4452 times

dap
Posts: 19
Joined: Mon Mar 09, 2020 6:38 pm
Has thanked: 27 times
Been thanked: 2 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by dap » Mon May 25, 2020 2:57 pm

Maybe I found the problem (I hope) - The windows WMI service reset seems do the trick
But could be a combo of many things (winmgmt reset, r pro 16.10 automatic update from today and memorial day low volumes)

On 10:47 (cest) I saw a fast cpu spike 100%, it lasted milliseconds, nothing longer for now.
From 17:00 to 18:30 I saw a lot of mini delays about 0,6 to 2 sec, warned by bookmap, but no spikes on cpu core only used by r trader (neither "slow computer" alert by r trader)

--

WARNING by Microsoft: Under any circumstances, do not delete the WMI repository as a first action because deleting the repository can cause damage to the system or to installed applications.
https://docs.microsoft.com/en-us/window ... dk/winmgmt

I did these steps without damaging my system or installed programs. 

Steps
1)close bookmap and r trader
2) open a cmd as administrator
3) launch net stop winmgmt
    stops the WMI Service. Respond in the affirmative to the warning message about stopping the Security Center and IP Helper (type Y and hit Enter)
4) launch winmgmt /resetrepository
    this will report the WMI repository has been reset
5) launch net start winmgmt 
    it may report the WMI Service is already started (that’s fine)
6) reboot windows    

source: https://us.forums.blizzard.com/en/wow/t ... er/37733/3
--


This "winmgmt" reset also improved battlenet client downloads performance which, (in the last weeks) was unusable with cpu0 at 100% and windows almost freezed during downloads. Also firefox improved/decreased cpu usage
But I didn't thought there was a correlation

Will test it in the next normal trading days



I also found another last chance to try, BES which limit the CPU for a selected process. But in this case I don't think would be a good solution, probably will worsen the r trader delays (http://mion.faireal.net/BES/)

 

Svyatoslav
Site Admin
Posts: 88
Joined: Mon Jun 11, 2018 11:44 am
Has thanked: 2 times
Been thanked: 7 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by Svyatoslav » Tue Jul 07, 2020 2:48 pm

Hi everyone,

We are still trying to pinpoint and handle this kind of issues, if you'd like to help - check out the experimental build in https://www.bookmap.com/forum/viewtopic ... 268&p=2755
Even if it does not help, it will at least give us some additional information.

Thanks,
Svyatoslav

TomB
Posts: 30
Joined: Fri Dec 14, 2018 1:34 pm
Has thanked: 4 times
Been thanked: 5 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by TomB » Tue Jul 07, 2020 4:44 pm

Hi: This might seem basic since my level of understanding is basic but I had similar issues with continual latency in active markets and 10-20 sec delays and found the issue, at least for me, was Cpu throttling. I am running a I-7 3.40Ghz, 32Gig Ram Win 10 64 Bit. 4 monitors + R-Trader Pro connected to BM and Investor RT.

My Cpu temp would spike to 95 deg c. I had checked everything with Task Mgr and it did not show excessive CPU load - high occasionally but not extreme - but no temperature. I downloaded a free program called "Speccy" which can monitor components. It showed the spikes in CPU heat. when the market was active and I was getting the latency. This included the open/close and stop runs. I found upon inspection that my CPU Heatsink was loaded with dust. I did a thorough removal of dust and this helped significantly - including cleaning a filter under the case which I didn't know was there. In addition, I added an air cooler. Now I rarely experience any latency and my CPU, when stressed, runs about 55 deg C under fast market conditions vs 95 Deg previously.  Hope this is helpful. TomB

 

dap
Posts: 19
Joined: Mon Mar 09, 2020 6:38 pm
Has thanked: 27 times
Been thanked: 2 times

Re: Rithmic latencies - HW,windows or rithmic? possible workaround

Post by dap » Thu Jul 09, 2020 3:36 pm

Svyatoslav wrote:
Tue Jul 07, 2020 2:48 pm
Hi everyone,

We are still trying to pinpoint and handle this kind of issues, if you'd like to help - check out the experimental build in https://www.bookmap.com/forum/viewtopic ... 268&p=2755
Even if it does not help, it will at least give us some additional information.

Thanks,
Svyatoslav
Ok I will try it.
But in the last couple weeks (more or less) I didn't have delay > 2 seconds. Don't know if it will help testing the new build

Instead 3 or more weeks ago I had to change the affinity cpu. I noticed (I think) the big delays appeared when r trader had 2 logic cpu to 100% (never seen 100% spikes on more than 2 logic cpu)

PS: From the last 3,4 weeks I use bookmap + overcharts both connected to r trader pro
 

Post Reply