Bookmap Performance FAQ

When I enter a symbol in Bookmap it keeps connecting. What’s the problem?

This indicates the symbol you entered is not recognized. This could be either due to entering an expired contract, wrong symbol syntax or symbol you have no data subscription for.

When I add a symbol, the symbol button remains greyed out

Most likely you subscribed to the free offline plan. Free offline subscription does allow recording of live depth data, for a limited period, but it does not allow real-time viewing of the data. So the data is recorded but the symbol buttons remain inactive.

The recorded data can be viewed with Bookmap replay after the recording session ends. If you wish to switch from the free subscription to the S5 special free live trial, please contact us from the support tab.

Bookmap takes a long time to connect (or does not connect at all)

Sometimes there can be certain routing issues or certain data sources can experience difficulties. While the solution is to wait for the data source to fix the problem (or for some ISP to fix routing issues), you can often work around that by picking another connection point, if data source provides such option.

E.g. for Rithmic, you can try selecting Europe or the US even if it's on the other side of the globe, as that will force traffic to take another route and will avoid some routing problems or issues if those are not affecting all data centres.

Bookmap gets slower over time

Start with removing all external indicators and restart Bookmap. Some might be storing data within java heap and after some time garbage collection has to kick in too often. For indicator developers: it is a bad practice to store massive amounts of data in RAM. If you want to save a copy of all incoming data for further processing, make sure you are saving it on disk and not in RAM.

Market data is lagging

Take a look at how exactly data is lagging:

If the timeline is moving consistently (matches your system clock), it probably is an issue with receiving the data. Make sure you are connected to the nearest connection point (e.g. Rithmic provides you with a choice) and make sure your internet connection is good. As a last resort you can try unsubscribing from some instruments, but that's just a way to avoid the problem, not a way to solve it.

If timeline lags behind (i.e. the time within Bookmap is behind your system clock) and moves inconsistently (getting faster and slower), the problem lies within the Bookmap process. The first thing to try is removing all external indicators (and restarting Bookmap afterwards). If the issue continues to occur contact Bookmap support: support@bookmap.com

Data lagging with Rithmic

With Rithmic you get a separate warning when data lags before getting processed by bookmap. That usually implies bad internet connection or problems with R Trader, if R Trader is used.

Rithmic uses single TCP connection in order to receive data. That means it's very sensitive to package loss. Things like unstable connection (e.g. DOCSIS with bad cable) or wifi can easily introduce enough package loss to limit per-connection speed too much. Also just having a bad internet provider or even a bad route (bad node between you and gateway) can impact your experience a lot. The easiest thing to try is connecting without R Trader to the a different gateway that is close to you. If it helps - try switching back to plugin mode (if you need it) but with different gateway in R-Trader.

When using R Trader (plugin mode) it's important that it is not starved of resources for whatever reason. We did see it on old Ryzen's probably due to windows scheduler issues. There is a topic on the forum where user presumably solved it by setting affinity (locking which cores R Trader can run on).

Having fast internet connection does not mean having low package loss. You can have 1GBit connection but still lose 0.1-1% of packages. In this case you might even reach your full 1GBit when downloading torrents (or doing something else that uses many connections at once or is optimized for package loss) but will still have bad experience with Rithmic.

Data lagging with NinjaTrader

The problem I'm having is that Ninjatrader lags pretty bad when there is significant movement in the markets. By lag, I meant the NT dom lags (can't move or place orders, price doesn't update for 2-3 seconds) as well as NT charts not updating promptly. Bookmap seems to be running fine with no lag while this is happening. If I remove the Bookmap indicators from the three charts in Ninjatrader then the lag no longer happens.

Make sure that both Bookmap and NT are set with equal priority as shown below. It’s highly recommended to set both with Normal priority. Based on your description it’s likely that Bookmap has higher priority than NT. Bookmap itself never sets or changes those settings.

The default priority is Normal, set by Windows OS. However, it happened in the past that Bookmap users change these settings manually and then forgot about it. If this is not the case and you still have this problem, please contact us and tech support will help.

Data is lagging for other reasons

Bookmap is set to "Normal" priority. Ninjatrader is also set to Normal, I never changed these values. I am running Windows 10 btw. I still experience the lag. Again, this lag is usually apparent under heavier market activity. At this time I'm only running CL and ES on Bookmap.

Starting from Bookmap 7 and above it appears in the task manager as bookmap.exe instead of java.exe.

The next candidate for the problem source is your Antivirus (specifically host intrusion prevention system, sandbox) that may cause the lag. Try to disable your antivirus.

If this doesn't help, update NT7 to NT8. NT8 seems to run indicators in a separate thread, not in the UI (user interface) thread. So, even if communication between NT and Bookmap isn't fast enough to handle spikes in data activity, the UI of NT8 will not freeze. This at least would be a workaround.

In general, note that NT doesn't have a standard API exposed to other processes. It's intended only to indicators (NT scripts), running inside NT process. But because Bookmap is itself a standalone program, it cannot be rewritten as NT indicator. Instead, when you add a Bookmap indicator to NT, this indicator communicates with Bookmap through Windows interprocess communication protocol. While it's pretty fast and we've done a lot to optimize it, it's not as fast as direct connections as with the rest of platforms and API's that Bookmap supports.

Also, this interprocess communication can be blocked or slowed down by Antiviruses as false positive detection. If the above didn't help, write to Bookmap support and request remote desktop support referencing this page.

Bookmap Cache folder and its size

I did notice that the C:\Bookmap\Cache\ folder grows to about 2.5 GB by the end of the day. Is it is normal?

Yes, it is normal. Market data typically consists of 90-97% of market depth updates (order book) and the rest is trades data. Besides, this is one of the reasons why observing market depth is so important. But that also implies technological challenges. Most trading charts don’t need to handle market depth data and even the trades data is aggregated into candlesticks. Even if those charts display DOM, they need to handle only the current snapshot of market depth data.

Bookmap chart may contain millions of market data events. The technological challenges are to allow smooth real-time chart display like HD video experience and also to allow quick and seamless chart manipulation. It required fast access to the market data collected so far. Handling all this data in RAM is not an option due to the large amounts of the data which can reach many gigabytes before compression.

To solve this Bookmap has developed a proprietary database and a data structure that effectively use both RAM and disk drive. The Bookmap\Cache folder contains the latter. It can easily grow to several gigabytes and it’s normal. You can roughly estimate the expected structure size as 1KiB for every event in the data you receive. The proprietary data structure is optimized for fastest access with both SSD and HDD, but SSD is obviously faster.

If you're willing to store your cache somewhere else (on another disc for example) you need to reinstall your Bookmap and select another place for installation. If you're willing to save all your settings you need to replace all contents of Bookmap folder (C:\Bookmap by default). Do not forget to save your workspace and open it in a new instance.

Installing Bookmap on SSD drive

There are two main folders which Bookmap creates during installation. Users can change the default location.

• Installation folder. By default: C:\Program Files\Bookmap\
• User folder. By default: C:\Bookmap

From the performance point of view, it doesn't really matter on which drive Installation folder is. But if there is enough space on the SSD drive, it’s recommended to install the User folder on it because this is where market data is maintained by Bookmap and SSD is faster than HDD.

Graphic Video cards and GPU

GPU’s are Graphics Processing Unit - the core of graphic/video cards. Unlike CPU that is optimized for computations, GPU’s are optimized for quick and smooth rendering on display, thus reducing the load on CPUs and RAM. Bookmap utilizes modern GPU technologies and uses OpenGL 3.0 interface. OpenGL 3.0 is available since 2008, so most computers are supporting it. Users with older graphic cards can disable it via Settings->GPU acceleration. Another feature is Settings➝Enable smoothing. This allows a smooth transition of chart display on the sub-pixel level. It’s mostly noticeable on low-resolution monitors.

Today many computers have both integrated and dedicated (typically more powerful) graphic cards. Users can let Bookmap use the more powerful graphic card as follows:

• Right-click on your desktop, and open NVIDIA Control Panel.
• Select “Manage 3D settings”, find Bookmap in the list of programs, and define which graphic card it should use.

Memory or RAM consumption of Bookmap

Bookmap maintains large amounts of market data. While most of the data is located in Bookmap/Cache folder, some part of it is managed in RAM for even quicker access. A typical memory consumption that appears in Windows’ Task Manager is 1.5 GB.

It doesn’t mean that all this memory is being used by Bookmap, but this is how much is allocated for Bookmap. Technically Bookmap uses less memory, but reducing the allocation limit would make Garbage Collector (Bookmap is a Java program) optimize memory size more frequently, thus slightly interrupting the smooth operation of the program.

Bookmap 7 (released on 16-Jan-2018) can automatically set maximum RAM allocation based on computer hardware. Users can adjust the settings manually via Settings -> Memory allocation:

Expected CPU load

Typically Bookmap consumes 2-10% of CPU depending on computer hardware.

It’s highly recommended to keep hardware drivers updated or to make sure they are configured for automatic updates. This is especially important for newer hardware. Keep Windows OS up to date is also important. For instance, recently Desktop Window Manager (dwm.exe) was causing a significant memory leak, but later it was fixed with one of Windows updates.

CPU consumption is expected to be higher during:

• Chart manipulations such as shifting, zooming, adding indicators or changing their settings, etc.
• Starting from Bookmap 7 -- loading historical market depth data after subscription to an instrument.
• Loading data in Replay mode (see the progress bar in Replay controls panel).

If you experience a higher CPU load by Bookmap in other situations, please contact our support for assistance.

Additional factors that can increase the CPU load:

• Not using GPU acceleration or old and low memory graphic card.
• Very high-resolution displays.
• The number of Bookmap charts detached for a separate view.
• The number of subscribed instruments/tabs.
• High-speed Replay mode.
• Increased chart refresh rate. Set larger refresh period (means lower FPS - frames per second) via Settings ➝ Chart refresh rate.
• Zooming out. The more price levels are displayed on the chart and the longer period - the more computations are required.
• The number of indicators displayed on the chart.

How Can I Increase the Performance of Bookmap?

If you encounter performance issues, it could be caused by your market data provider, Bookmap, your network/connection, or by your hardware.

Here's how we suggest you handle performance issues:

1. If you see red text on the upper left of the Bookmap chart informing you about data latency, it means that the speed at which you can receive data is limited.

This speed is affected by package loss, therefore decreasing the package loss will help the speed. Consider implementing one or more of the following:

• Decrease the number of instruments you are subscribed to.
• If you are connected via R-Trader pro (plugin mode), try connecting directly.
• If you are using wifi, switch to cable.
• If you have multiple internet connections, try switching to another one.
• In the case of a Rithmic connection, consider switching the Rithmic gateway that you connect to. We also received some reports that simply reconnecting can help.

2. If you see bookmap "freezing" (where you try to recenter the chart, but it does not update), please consider:

• Decreasing the number of instruments that you use. Closing some of the charts might help, however, it's best to simply unsubscribe from the instrument.
• Check the add-ons that you’re running. Consider unloading those and comparing the load. Some add-ons could perform heavy operations on the main thread blocking it (that's not how add-ons should work, but that's how some add-ons might!).
• Try enabling "Experimental multithreading" under the Settings > Performance menu. It should boost processing speed by about 15-20%. - Make sure to allocate enough RAM under “Settings”. 8096MB should be more than enough.
• If you are running Bookmap 7.4 or higher, you can try enabling “Experimental Serialization” under the Settings > Performance menu (just under experimental multithreading). This function is designed to increase Bookmap performance and speed across the board. Please note that “Experimental Serialization” is a new feature, and as such traders should use this mode with caution. Extra care should be taken during live trading due to limited user testing of Experimental Serialization
• Make sure "Automatic aggregation" is enabled under the Settings > Performance menu by following the on-screen instructions. (This is usually on by default)
• Make sure you have allocated enough RAM for Bookmap in Bookmap memory allocation settings (especially important if experimental multithreading is turned on). On Mac/linux you have to tweak corresponding files instead.
• Consider trying Bookmap 7.4 as it has additional performance improvements. For some use cases (many charts with many indicators, all open at once) you can see a 25% improvement - which is quite significant.
• For those who are using Rithmic and don’t need the MBO data (if you also don’t use Bookmap’s Stop & Iceberg Tracker or Liquidity Tracker), you can switch on the aggregation. If you are connecting through R-Trader Pro, make sure to enable it in R-Trader, not just in Bookmap. It should help with both of the above.
• If you are running older or less powerful hardware - consider upgrading (find hardware requirements); keep in mind that MBO with Rithmic requires more memory and processing power.
• Feel free to experiment with turning parts of software and indicators on or off and comparing the utilization.

We hope this will help. If you still have issues or questions, please reach out to support@bookmap.com