Having multiple memory issues daily. Need help,

[Svyatoslav]

What you describe sounds like bookmap starts running out of memory (specifically the CPU usage climbing). Until we fix the addon memory leak - I'd recommend to just restart.

[dmavro]

Just saw your last post.... 

> does it make sense to you that i had to go down to 3 subscriptions since i started with BM all those months ago?

Generally main reason to do so would not be memory but ensuring bookmap is able to keep up with the data. Depending on addons used, computer specs, and specific instruments you are watching, you might or might not be OK with more instruments. Also, current level of market volatility plays a big role - in volatile days having less instruments might help a lot, but when market is calm you might be ok with many instruments.
I'd say that if you are having memory issues with ~10 instruments while having 8+GiB allocated and are not running something extreme (e.g. setting BTCUSD pips to 0.01 is basically a guaranteed way to run into issues sooner or later, while something around 10 or 50 should be pretty safe) - it's worth further investigation. That being said, 10 instruments with MBO data could cause performance issues.

I really thought the build im using would have let me get away with more then 3 subs. I guess im using alot. Heres what ive been using...
I’m basically using everything in study configuration pane except for Candlestick and Strength indicator. Then in addons I definitely am using a lot. I’m using all the MBO addons (both S&I’s, Absorption, Sweeps) plus Liq Tracker Pro(just viewing Diff line.) I’m also running 4 of my own addons(with no history) that just read from CSV files and plot lines all day. Some are static for the day and some update constantly but they all read from their CSV files and update at 5, 10 or 15 second intervals. Tradestation also writes to those CSV's all day long. Some at the end of 1minute bar intervals and some every 5 seconds. Not sure if i might have to much writing/reading going on there but doesnt seem to be an issue. TS mem use is always around 1gb for all processes and CPU processes are always low. 

Does that sound reasonable to you for 3 subs only or do you think i should be able to get a more out of my system? 

Thanks,
Dean

 

[dmavro]

What you describe sounds like bookmap starts running out of memory (specifically the CPU usage climbing). Until we fix the addon memory leak - I'd recommend to just restart.

Will do! Thanks!

[dmavro]

Svyatoslav,

Wanted to ask a couple more things...

1. How will i know when memory leak is resolved? Will i be notified here or will you guys put update on Discord channel?

2. Would previous versions on S&I on chart also have leak? I restarted yesterday at 1pm due to CPU running away. And at one point after 3:45 when i walked away from computer i got memory error again. I sat down at desk again around 5:30 and saw it on screen so it had to have happened to me in about 3-4 hours that time. I know it was FOMC day but now i shut it off for today and I would really like to use one of them if possible.

3. Still dont understand why my laptop didnt seem affected and why no other BM users are dealing with same issue. Why would that be?

4. Could you please look at my 2nd to last post from yesterday and let me know what your opinion is? 

No matter what my system runs like after you guys resolve leak. My ultimate goal going forward is to be subscribed to ES, CL, NQ and GC every day and have never been able to handle that in the past. Only reason i ever went to ES, CL and YM was due to the issues i saw when running the others together. So if memory leak has been around since i started and i do end up seeing a difference going forward then thats great but if system is going to still keep me limited to 3 subs then would upgrading to a new 12 gen i7-12700k or an 11th gen i9-19000k or even the i7-11700k maybe give me more headroom? 


Thanks,
Dean

[Svyatoslav]

> Does that sound reasonable to you for 3 subs only or do you think i should be able to get a more out of my system?

I'd assume you should be able to get more.

> How will i know when memory leak is resolved? Will i be notified here or will you guys put update on Discord channel?

The addon just passed QA. Let's see if it works. Please get an updated addon at https://bookmap.com/addons/si-onchart

> Would previous versions on S&I on chart also have leak?

Some do. Best to try the above link.

> Still dont understand why my laptop didnt seem affected and why no other BM users are dealing with same issue. Why would that be?

Well, that puzzles me too. Maybe different addon version there? Anyway, I suggest that you try on the desktop and if you see any issues - we'll just keep working on those. I'll need new logs and memory dumps if issues will still be there.

> if system is going to still keep me limited to 3 subs then would upgrading to a new 12 gen i7-12700k or an 11th gen i9-19000k or even the i7-11700k maybe give me more headroom? 

That will definitely give you more headroom, especially if you wait a bit and go for i9 12900k with DDR5 RAM as that combination presumably has very significant improvement in IPC (instructions per cycle). However 12th gen will require switching to Windows 11, afaik, since windows 10 scheduler does not know how to work with variations of big.LITTLE architecture (though maybe they'll improve win10 scheduler).

Still, if you only need 5 instruments - you might be able to do that on current system. Unless you were going to upgrade anyway, I'm suggesting that you try updated addon, and if the issues continue - we just keep working on locating/fixing those until we either don't see any obvious issues or we get it running.

[dmavro]

Svyatoslav,

Thank you for responding. I just downloaded and installed new verison. Would you recommend that i lower memory allocation now back down to the 8gb like i always had it or should i stay at 16Gb for the meantime? 

Also, my Laptop and desktop are setup the same with all the same addon versions. I actually made sure they were same when i used laptop for troubleshooting. Thats whats been bugging me so much. 

Those new 12 Gen CPU's look great on paper and ive seen some great benchmarks online. I knew you would have to upgrade to windows 11 to use it and I truly hate being a guinea pig for new OS's. Also, a bit worried about the new big.LITTLE architecture since its the 1st desktop processor they've attempted this in. Plus, it's impossible to find DDR5 Ram at moment and theres no reason in my mind to purchase one with DDR4 ram and mobo because to do that and then upgrade to DDR5 Ram down the road would require a new mobo too. So im really hoping this new version fixes my issue that way i can wait for some supply to hopefully hit the mkt next year at one point and for prices to come down. 

Have a great weekend.

Thanks again,
Dean

[dmavro]

I lowered RAM allocation back to 8GB level on Friday and added GC. I had no issues all day. I started BM last night and woke up this morning and had memory warning so i created heap dump. When i restarted BM i saw that i also had a critical memory error like in the past. I just increased RAM to 16gb again and restarted. Going to send heap dump to support in a few.

Thanks,
Dean

[Svyatoslav]

>Would you recommend that i lower memory allocation now back down to the 8gb like i always had it or should i stay at 16Gb for the meantime? 

I'd keep it at 16GiB. It makes it easier to distinguish "inefficient memory usage" from an actual memory leak.

> Going to send heap dump to support in a few.
Ok, waiting for the memory dumps, will update you once those reach me and I have a chance to investigate.

 

[Svyatoslav]

I think we are making some progress (SI on chart objects are gone), but now I see that some indicators ended up consuming ~3.7GiB. I'm not 100% sure if it's an issue in Bookmap or in the indicators themselves, but I'd like to locate indicator developers to discuss that directly, as that would be the fastest way.

Could you please help me... Do you know which addons out of what you use produce indicators named like:
- orBO.ORBO_v9
- priorStats.ONHL_YHL
- mab.MAB_v4
- IB.IB_v9

Thanks,
Svyatoslav

[dmavro]

I think we are making some progress (SI on chart objects are gone), but now I see that some indicators ended up consuming ~3.7GiB. I'm not 100% sure if it's an issue in Bookmap or in the indicators themselves, but I'd like to locate indicator developers to discuss that directly, as that would be the fastest way.

Could you please help me... Do you know which addons out of what you use produce indicators named like:
- orBO.ORBO_v9
- priorStats.ONHL_YHL
- mab.MAB_v4
- IB.IB_v9

Thanks,
Svyatoslav

Thats good news about the S&I. Bad news is those others are the ones i coded. The ones i mentioned in earlier post, that read CSV files constantly throughout day. They utilize a timer to update at specified seconds interval. I had never coded in Java before i started with BM so these were my first attempt at it. orBO is the 1st one I developed and probably does the most out of the 3(alot of Lines drawn for day and has some local time conditions that stop and restart plotting in AM) and then when i started on the next 3 I used that as a template. 

Dean