Allow Full Reloading of Cached Data NOT Partial Only

Add your new features and requests here.
magicdonkeyempor
Posts: 28
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Allow Full Reloading of Cached Data NOT Partial Only

Post by magicdonkeyempor » Thu Feb 10, 2022 2:53 am

Every so often bookmap crashes on me.

I keep my computer on all week long to horde data.

When this happens at the end of the week it's massively incredibly frustrating as I've lost the data which I need to log.

Currently you have an arbitrary limit of ONLY 22hrs data reloading from cache after a crash

So Simple Request:

Untitled-1.jpg
Untitled-1.jpg (26.46 KiB) Viewed 9852 times

Please lift the limit on this so I can reload ALL data stored locally on my computer after a crash!

magicdonkeyempor
Posts: 28
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by magicdonkeyempor » Tue Feb 15, 2022 11:25 pm

Another data dump crash today in the middle of the week.

To say I'm frustrated is an understatement.

Please get this lifted!

Svyatoslav
Site Admin
Posts: 278
Joined: Mon Jun 11, 2018 11:44 am
Has thanked: 2 times
Been thanked: 31 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by Svyatoslav » Thu Feb 17, 2022 9:13 am

Hi,
We are internally discussing a possibility to do what you requested. In the meantime, I'm interested in why it runs out of memory for you. How much memory do you have allocated? If it's 8GiB of more - could you please click "create a heap dump" button on that window and send it to our support team (ask them to forward it to me).
Thanks,
Svyatoslav

magicdonkeyempor
Posts: 28
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by magicdonkeyempor » Thu Feb 17, 2022 9:56 am

Hi Svyatoslav,
Many thanks for your update & very much appreciated to hear that you're looking at it internally.

I have allocated ~9GB of RAM in Bookmap
on v7.3.0 b24

Usually running with around 6GB free memory out of 32GB total system RAM.

With the heap dump -> do you mean "Core dump on crash"?:
core dump.jpg
core dump.jpg (41.64 KiB) Viewed 9798 times

If so, seems to say that it's several GB file size, and only on crash.

michael_loboyko
Posts: 21
Joined: Wed Aug 04, 2021 11:52 am
Been thanked: 8 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by michael_loboyko » Wed Mar 16, 2022 9:26 am

Hi magicdonkeyempor,

Recently, we've increased local data import from 24h to 7 days for Global and Global Plus packages. Please check.

Regards,
Michael

magicdonkeyempor
Posts: 28
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by magicdonkeyempor » Mon Apr 04, 2022 9:35 am

michael_loboyko wrote:
Wed Mar 16, 2022 9:26 am
Hi magicdonkeyempor,

Recently, we've increased local data import from 24h to 7 days for Global and Global Plus packages. Please check.

Regards,
Michael
Hi Michael,

Just seen your message, would be good if the forum had an email notification or discord ping.

In any case, just upgraded to v7.3.0 b25 today, will give it a test at the end of the week.

Much appreciated for the update!

magicdonkeyempor
Posts: 28
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Allow Full Reloading of Cached Data NOT Partial Only

Post by magicdonkeyempor » Mon Apr 11, 2022 12:07 am

Hi Michael,

Just done the test on the new week.

So I can load data 7 days back...
but turns out that the data is completely messed up & unusable:
messed up data.jpg
messed up data.jpg (372.81 KiB) Viewed 9597 times

Sample above over a 4hr period - looking back 7 days ago

Seems that it's no longer order flow data, but intermittent chunked data
 - chunks are 1x every 10minutes
 - profile as you can see is complete rubbish & inaccurate for my practical use
 - and heatmap doesn't accurately show changes in levels (only a chunked snapshot)

I still appreciate the progress that we can load back 7 days now

But please restore proper loading of continuous flow data

Post Reply