Page 1 of 1

Bookmap lag

Posted: Mon Jan 06, 2020 4:45 pm
by 1zach4
Having issues with lag today.  It started out as an error with loading data from the cloud, and then after updating to the latest version I started getting gaps in depth, and trade dots/bid-ask lines would lag each other.  Then the bid/ask and trade dots would line up, but the data was 5-10 seconds behind my other platforms.

Anyone have any experience with this issue in the past?  Using dxfeed for CME futures markets, issues are showing up on both CL and ES

Re: Bookmap lag

Posted: Tue Jan 07, 2020 2:54 pm
by 1zach4
Still having major issues with bid/ask lines and trade dots being separated by 10-15 ticks on cl/es futures, and still lagging significantly.  Bookmap support says it's dxfeed, but is anyone else having this issue?

Re: Bookmap lag

Posted: Wed Jan 08, 2020 12:41 am
by bronco33
I have been having the same problems, huge data gaps, lags, will only load history up to 30 seconds. I have been told the last two days it would be fixed but nothing has been done. Is there an alternative to DXfeed?

Re: Bookmap lag

Posted: Wed Jan 08, 2020 2:07 pm
by s54mcoupe
I'm having the same problems as well.

Re: Bookmap lag

Posted: Thu Jan 09, 2020 8:40 am
by Svyatoslav
Hi everyone. There seem to be 3 different problems here:
- Historical data not downloading (timeout)
- Consistent data lag (trades/depth/BBO delayed consistently)
- Data desync (trades/depth/BBO desync)

First two are typically network-related. It's not about your network specifically (but about it too), it's mostly about the highest speed TCP connection between you and data server can have (which is affected by ping and package loss, and those two are often more important than your connection max speed). Thing is that DX servers providing data to Bookmap are located in Europe, which is a bit of a problem since your connection is very "long range" if you are in US, resulting in high package loss and ping as soon as any link in the chain starts having issues or gets overloaded.

The third one is new and we are still not sure what's causing it. Guesses are that it might be related to the same connection speed and/or local computer performance, but we don't have any certainty so far. There are other potential causes too.

In order to help us resolve the issue, you can do one of the following (and I'll be really grateful if you do):
- be very specific about which of those 3 problems each one of you is experiencing. Understanding precise impact lets us guess the underlying cause better, and we need to know it before we can mitigate the issue.
- provide me your IP address. This will let us check the logs on DxFeed side and see if there is anything suspicious visible from their side. Ideally do the following: connect to DX, wait until problem starts, then check your ip (e.g. by typing "my ip" in google). Send it to me along with description of which issues out of those 3 are affecting you specifically and when exactly that was (time down to minutes) (you can post IP here, if you want, you can send it to [email protected] or PM me - whatever you prefer)
- if you are experiencing the 3rd issue (desync) and can more-or-less reliably reproduce it - I will very much appreciate if you allow me to remote to your computer. I want to check how exactly it looks (you can send a video for that, as an alternative), but I'll also try to see how good connection to DX servers is and maybe will figure out what else can be done (e.g. one test we can run is rerouting the traffic a bit). For that please contact [email protected] and mention that Svyatoslav asked to arrange a remote with you in regards to DX data issues.

So, we need your help here, as we can't reproduce it on our side so far.
Sorry for the inconvenience, and we are hoping for community assistance in resolving this.

Re: Bookmap lag

Posted: Tue Jan 28, 2020 9:55 am
by Svyatoslav
Hi everyone. Just wanted to provide an update. We are still working on the issue, here is how things look now:
- We are testing US infrastructure that should speed up historical data download for US clients (and hopefully alleviate some latency issues too)
- We keep working with DX to resolve the issue of data quality. Unfortunately we keep experiencing an issue where trades get significantly delayed on server side (at least that's how it looks). US infrastructure might help with that to a degree too (at least in some cases we don't observe the issues there).
- NBBO data is also known to get desynced on server side. Orders (depth data) arrives first (sometimes up to few seconds faster), so if you want the most accurate BBO - that's what you should use (you can toggle that in visibility menu). NBBO data can lag.
- We hopefully fixed https://www.bookmap.com/forum/viewtopic.php?f=19&t=581 in latest builds. It is totally unrelated and fixes itself as soon as you zoom, but since it's visually similar thought it's worth pointing out, since these two bugs can potentially look like the same one - they are not.