Binary sbn #1486
Binary sbn #1486
Conversation
|
In the pull request is missing storing StrokeOptions. I did not noticed that it is another package and accidentally clened it. I must create code again. |
|
now commit is completed and can be tested On my Samsung Tab S6 is note which was read 10 seconds now read in much less than second. |
|
Have you compared file sizes? |
|
Normal file is 32 MB, binary file 16 MB. File contains only strokes. I used 4 bytes format of number instead of 8 bytes to store position. |
|
I am a bit confused. If I understand correctly, the current files are saved using |
|
I do not save double (8 bytes) but scaled integers (4 bytes) to save memory. 4 byte integer can have value +-2147483648. I take double multiply it by 1000 and save this number rounded to integer. I suppose that precision of 0.001 point is sufficient. Still I can handle canvas about 2147483 points (saber use 1000x1400 by default) so much more than needed (precision can be improved by multiplying 10000 or 100000 with sufficient size of canvas 21474 points and precision 0.00001). I do not use floats because if I have canvas of size 10000 points I have resolution of only 0.01 point. I used this approach to have smaller file size because each stroke point is 2 doubles (or 3 in case of pen pressure). This simplification save half of file size. The main reason of fast reading is that original code uses bson -> json -> data structures and it is slow (transfer to json). My method use binary file -> data structures. Using Json is nice when You want to be able to read newer file versions (more features) in old Saber. It interprets only known features. |
|
I have no idea about current infrastructures, but at least decades ago, the performance for integral arithmetic is much better than float arithmetic.
Is |
|
I use integers only for storage. during reading they are recalculated to doubles because Flutter uses doubles to draw objects. Json is combination of ASCII and doubles. It in fact list of different values. But it is converted to/from bson using strings. It is elegant to work with it, but it is slow. All data structures are objects. They must be read/stored using its methods storing/reading only important data (positions of stroke points, line type, color, ...). |
Do you mean that, if one wants to convert some data structure to I mean, it looks like that you could keep the current |
|
In addition, it looks like that, even for the original |
I did not studied original code too much, so did not care about bson (I used it to store only "text editor" - quill - stuff to String and then write it to file). I simply wanted to store data directly to file without any intermediate structures and see if it is faster. May be I do not see further consequencies and my code is wrong, but we at least know, that reading notes can be faster. I think that using Json as intermediate causes slow reading of notes and it should be managed differently. It is on @adil192 to decide how to implement it (if at all). If it will not be implemented, I will use my draft for my personal use. |
This commit implements writing note as binary file. It does not convert note to Json and save Json so especially reading of note is much faster.
When reading note file Json is also not used so reconstruction is faster.
(Json is internally only used to save/read quill, i.e. editor).
There is not solved versioning of note file. It means when new version of note will be read by older version of Saber, it will fail to read.
The purpose of this commit is provide alternative way to save/read note and improve speed of reading.
Int the preferences is check box "Save as binary" so this feature can be switched on only for testing.
Floating values of stroke points are stored as 4 byte integers (float multiplied by 1000, and rounded). It saves note size and provides sufficient precision of 0.001 pt.