Backfilled Data Not Accurate | Affects All Levels

Add your new features and requests here.
magicdonkeyempor
Posts: 30
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Backfilled Data Not Accurate | Affects All Levels

Post by magicdonkeyempor » Thu Jun 09, 2022 12:24 pm

When loading backfilled data after a crash, the data is aggregated into extremes.

Please do not allow this false aggregation, as it gives a poor view of levels, leading to bad trade decision making on my part.

There are countless examples when you scroll back.
For reference here is one from GCQ2 on 06/Jun/2022 @ 09:35am (+8 UTC):
2022-06-06-09-Backfilled-Data-Mess.gif
2022-06-06-09-Backfilled-Data-Mess.gif (557.48 KiB) Viewed 5499 times

Clearly backfilled data is completely different from what actually happened.

Note: Live data is correct & matches with the footprint (showing the issue lies with the backfill)
Live Cache Footprint Comparison.jpg
Live Cache Footprint Comparison.jpg (118.79 KiB) Viewed 5502 times

I've noted this issue in a similar fashion here:
 - where FULL flow data is not displayed
https://bookmap.com/forum/viewtopic.php?f=21&t=3517

Please do not allow this false aggregation.
 - (or at the very least provide an option for users wishing to backfill with compressed, aggregated data)
 - No aggregation should be the default
Last edited by magicdonkeyempor on Mon Aug 29, 2022 2:59 am, edited 1 time in total.

magicdonkeyempor
Posts: 30
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Backfilled Data Not Accurate

Post by magicdonkeyempor » Mon Aug 29, 2022 2:53 am

Another example from NQ which shows the difference between backfilled & live data:
 - this time from the cloud (vs previously local cache)
rithmic backfill issue.jpg
rithmic backfill issue.jpg (516.03 KiB) Viewed 5332 times

Note that the aggregation at incorrect levels,
Which makes backfilled data useless & at worst misleading.

Orderflow becomes completely unreadable for manual traders
 - and erroneous for automated traders (poor quality data)

No aggregation on backfilled data needs to be the default option
 - if other users wish to save on RAM with lower grade aggregated data, then that should be a separate option

magicdonkeyempor
Posts: 30
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Backfilled Data Not Accurate | Affects All Levels

Post by magicdonkeyempor » Thu May 30, 2024 2:01 am

Just to update,

This ugly chunking leading to erroneous data/ profiles is still present on V7.5.0 b19
 - specifically on backfilled cloud data

Backfilled data from locally cached history appears to be more smooth & accurate now.

Please fix chunking/aggregation on cloud data!
 - it is vitally important to have an accurate view of order flow & correct profile levels!
Backfilled cloud data aggregation erroneous.jpg
Backfilled cloud data aggregation erroneous.jpg (443.23 KiB) Viewed 3284 times

magicdonkeyempor
Posts: 30
Joined: Sun Feb 06, 2022 5:33 pm
Has thanked: 18 times
Been thanked: 4 times

Re: Backfilled Data Not Accurate | Affects All Levels

Post by magicdonkeyempor » Thu Jul 03, 2025 2:46 pm

3 years later and still looking for a fix for this

Ugly chunking is still there on backfilled cloud data in Bookmap V7.6.0 b26

Footprint Not Matching.jpg
Footprint Not Matching.jpg (142.85 KiB) Viewed 45 times

Backfilled cloud data should match the refresh rate and smoothness of live & locally cached data

If really an issue would prefer an option to recieve no data at all or smooth tick by tick data.
 - Should be no reason for anyone to pay for longer backfilled data when this sort of chunking is the result

Post Reply