How to fix degrading performance as an entry gets longer?

Asked by Rick Morris

First, thank you for a wonderful software. I'm having some intractable issues that I've ignored for some time due to the hassel of having to register yet another account, that I will infrequently use, to just post to this list...Anyways, I hope you can help with these problems.

1/Once an entry reaches a certain count in characters, say more than half a page, the scroll rate (arrow keys) becomes drastically reduced. This happens sooner, under 1000 words, when there are a lot of formatted text like url, lists, comments, images, etc. Sometimes, the scroll rate by mouse is also affected. Is there something I can do to ameliorate this? What is causing this? The program is having difficulty parsing all of the formatting? I notice that it will do realtime update to the text as the cursor runs through the text to expose the formatting fields. Perhaps it is over-doing things? Perhaps it should only expose the formatting when then cursor has stopped?

2/Search is exceedingly poor in performance. Typing into the search box causes UI freezes as the program looks through all of the texts available upon each keystroke. The UI performance is not improved when limiting search to just a subset of tags. Is there a way to limit search to just the current open entry? Perhaps search should pre-index the diary upon log out or at set interval? Maybe decouple UI update from search and have a separate area that the user can click on to navigate through the search results? Maybe only execute search once input has ceased for a specified duration?

I'm using the software on Gnome under Wayland. Maybe Wayland just doesn't like to play nice?

Question information

Language:
English Edit question
Status:
Answered
For:
Lifeograph Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Ahmet Öztürk (dmxe) said :
#1

Thank you for the report. Please also specify Lifeograph version you are using.

1) Lifeograph is not particularly good at dealing with really huge entries and I really recommend splitting entries into multiple ones after a certain point. That being said, 1000 words is definitely not too big. I myself can go up to 2000 words without any serious lag. Actually, Lifeograph's parsing system is more or less optimized not to waste any CPU power but there is still room for improvement, of course.
Coincidentally, nowadays, I was working on a new and improved parser --mostly in terms of capabilities though, rather than speed. I will also investigate ways of improving performance.
All in all, I suspect that Wayland has also something to do with this slowness. If you could try it open this long entries on X and compare, it would be really helpful.

2) Search is quite optimized in Lifeograph. In a diary with more than 2000 entries it can search in real time quite easily. I think the same problem as above hits again here. To be able to highlight the matches Lifeograph parses the current entry and if it happens to be long, search takes long. Please also try search in read-only mode. It should be faster then, except for transitions from entry to entry.

Revision history for this message
Rick Morris (mushroomancer) said :
#2

1/ My version is lifeograph-1.5.1.1-1.fc30.src.rpm. I initially installed it under Fedora 29 through Gnome Software, but it has since disappeared from the Gnome Software...I don't know why. However, I can still find it through pkcon or dnf, with the latter being used to update the system. Anyways, it's a separate issue.

2/ I compared scroll lag under Wayland and X session and under the latter it is even worse than the former for the same entry. The UI completely freezed as I held down the arrow key, which reappears somewhere down the entry some times later once the UI unfreezed. Under Wayland, the scroll is just very slow but does not cause UI freeze.

3/ I also note that there's no scroll lag when reading the entry, which is 1355 characters long and has quite a bit of urls, lists, and comments [[]]. So I copy and pasted the entry in question into vim to remove indentations and replaced all of the formatting fields with non-formatting ones like | for •, () for [[]], and https:// with |https://. I made a new entry with modified content and it still had the same scroll lag. Thus I concluded that the scroll lag is due to the parsing for formatting fields and trying to expand them in realtime needlessly. As the entry gets longer, predictive algorithm takes more time...It's not my place to tell you how to solve things, but perhaps only run the algorithm once the scrolling has come to a stop after some specified period?

4/ Search under read mode also causes UI freeze. I can't tell if the freeze is longer or shorter...maybe shorter?

Revision history for this message
Ahmet Öztürk (dmxe) said :
#3

Can you also specify the configuration of the computer you are getting these results? Especially the CPU.

Revision history for this message
Rick Morris (mushroomancer) said :
#4

/:-------------:\ ocelot@yellowtrain
       :-------------------:: ------------------
     :-----------/shhOHbmp---:\ OS: Fedora 30 (Workstation Edition) x86_64
   /-----------omMMMNNNMMD ---: Kernel: 5.1.12-300.fc30.x86_64
  :-----------sMMMMNMNMP. ---: Uptime: 1 day, 7 hours, 38 mins
 :-----------:MMMdP------- ---\ Packages: 2118 (rpm)
,------------:MMMd-------- ---: Shell: bash 5.0.7
:------------:MMMd------- .---: Resolution: 1920x1080
:---- oNMMMMMMMMMNho .----: DE: GNOME 3.32.2
:-- .+shhhMMMmhhy++ .------/ Theme: Adwaita [GTK2/3]
:- -------:MMMd--------------: Icons: Adwaita [GTK2/3]
:- --------/MMMd-------------; Terminal: gnome-terminal
:- ------/hMMMy------------: CPU: AMD Ryzen Threadripper 2950X 16- (32) @ 3.500GHz
:-- :dMNdhhdNMMNo------------; GPU: AMD ATI Radeon RX Vega 56/64
:---:sdNMMMMNds:------------: Memory: 7621MiB / 64345MiB
:------:://:-------------::
:---------------------://

Revision history for this message
Rick Morris (mushroomancer) said :
#5

System: Host: yellowtrain Kernel: 5.1.12-300.fc30.x86_64 x86_64 bits: 64 Console: tty 6 Distro: Fedora release 30 (Thirty)
CPU: Topology: 16-Core (2-Die) model: AMD Ryzen Threadripper 2950X bits: 64 type: MT MCP MCM L2 cache: 8192 KiB
           Speed: 2110 MHz min/max: 2200/3500 MHz Core speeds (MHz): 1: 2110 2: 2216 3: 2112 4: 2114 5: 2149 6: 2088 7: 2141
           8: 2140 9: 2202 10: 2530 11: 2159 12: 2080 13: 2611 14: 2403 15: 2203 16: 2087 17: 2689 18: 2515 19: 2123 20: 2136
           21: 2101 22: 2136 23: 2127 24: 2117 25: 2169 26: 2123 27: 2516 28: 2126 29: 2107 30: 2160 31: 3803 32: 2118
Graphics: Device-1: Advanced Micro Devices [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] driver: amdgpu v: kernel
           Display: server: X.org 1.20.4 driver: amdgpu tty: 150x35
           Message: Advanced graphics data unavailable for root.
Audio: Device-1: Advanced Micro Devices [AMD] Family 17h HD Audio driver: snd_hda_intel
           Device-2: Advanced Micro Devices [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] driver: snd_hda_intel
           Device-3: Logitech G933 Wireless Headset Dongle type: USB driver: hid-generic,snd-usb-audio,usbhid
           Sound Server: ALSA v: k5.1.12-300.fc30.x86_64
[ocelot@yellowtrain ~]$ sudo inxi -I
Info: Processes: 494 Uptime: 1d 7h 49m Memory: 62.84 GiB used: 7.39 GiB (11.8%) Shell: bash inxi: 3.0.34
[ocelot@yellowtrain ~]$ sudo inxi -F
System: Host: yellowtrain Kernel: 5.1.12-300.fc30.x86_64 x86_64 bits: 64 Console: tty 6 Distro: Fedora release 30 (Thirty)
Machine: Type: Desktop Mobo: ASRock model: X399 Taichi serial: M80-BA021500303 UEFI: American Megatrends v: P3.50
           date: 12/24/2018
CPU: Topology: 16-Core (2-Die) model: AMD Ryzen Threadripper 2950X bits: 64 type: MT MCP MCM L2 cache: 8192 KiB
           Speed: 2111 MHz min/max: 2200/3500 MHz Core speeds (MHz): 1: 2086 2: 2093 3: 2045 4: 2048 5: 2083 6: 2097 7: 2056
           8: 2091 9: 2095 10: 2105 11: 2105 12: 2107 13: 2095 14: 2092 15: 2089 16: 2101 17: 2112 18: 2096 19: 2072 20: 2034
           21: 2064 22: 2087 23: 2086 24: 2097 25: 2089 26: 2051 27: 2147 28: 2096 29: 2040 30: 2103 31: 2056 32: 2044
Graphics: Device-1: Advanced Micro Devices [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] driver: amdgpu v: kernel
           Display: server: X.org 1.20.4 driver: amdgpu tty: 150x35
           Message: Advanced graphics data unavailable for root.
Audio: Device-1: Advanced Micro Devices [AMD] Family 17h HD Audio driver: snd_hda_intel
           Device-2: Advanced Micro Devices [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] driver: snd_hda_intel
           Device-3: Logitech G933 Wireless Headset Dongle type: USB driver: hid-generic,snd-usb-audio,usbhid
           Sound Server: ALSA v: k5.1.12-300.fc30.x86_64
Network: Device-1: Intel I211 Gigabit Network driver: igb
           IF: enp4s0 state: up speed: 1000 Mbps duplex: full mac: 70:85:c2:ae:09:8d
           Device-2: Intel Dual Band Wireless-AC 3168NGW [Stone Peak] driver: iwlwifi
           IF: wlp5s0 state: down mac: 62:82:bb:fe:00:9c
           Device-3: Intel I211 Gigabit Network driver: igb
           IF: enp6s0 state: down mac: 70:85:c2:ae:09:8f
           IF-ID-1: virbr0 state: down mac: 52:54:00:a5:72:16
           IF-ID-2: virbr0-nic state: down mac: 52:54:00:a5:72:16
Drives: Local Storage: total: 244.47 GiB used: 12.80 GiB (5.2%)
           ID-1: /dev/nvme0n1 vendor: A-Data model: SX8200PNP size: 238.47 GiB
           ID-2: /dev/nvme1n1 vendor: A-Data model: SX8200PNP size: 238.47 GiB
           ID-3: /dev/nvme2n1 vendor: A-Data model: SX8200PNP size: 238.47 GiB
           ID-4: /dev/sda model: ANS9010B N111N1110 0 3150 200 size: 6.00 GiB
RAID: Device-1: md124 type: mdraid status: active raid: raid-5 report: 3/3 UUU Components:
           online: nvme2n1p5~c0 nvme1n1p3~c3 nvme0n1p3~c1
           Device-2: md125 type: mdraid status: active raid: raid-0 report: N/A Components:
           online: nvme1n1p4~c2 nvme2n1p6~c0 nvme0n1p4~c1
           Device-3: md126 type: mdraid status: active raid: raid-0 report: N/A Components:
           online: nvme2n1p4~c0 nvme0n1p2~c1 nvme1n1p2~c2
           Device-4: md127 type: mdraid status: active raid: raid-0 report: N/A Components:
           online: nvme2n1p3~c0 nvme1n1p1~c2 nvme0n1p1~c1
Partition: ID-1: / size: 19.56 GiB used: 289.1 MiB (1.4%) fs: ext4 dev: /dev/nvme2n1p2
           ID-2: /home size: 361.94 GiB used: 3.64 GiB (1.0%) fs: ext4 dev: /dev/md125
           ID-3: /usr size: 124.99 GiB used: 7.30 GiB (5.8%) fs: ext4 dev: /dev/md126
           ID-4: /var size: 60.74 GiB used: 1.57 GiB (2.6%) fs: ext4 dev: /dev/md124
           ID-5: swap-1 size: 63.85 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/md127
Sensors: System Temperatures: cpu: 31.1 C mobo: 31.0 C gpu: amdgpu temp: 33 C
           Fan Speeds (RPM): fan-1: 0 fan-2: 0 fan-3: 2083 fan-4: 754 fan-5: 788 gpu: amdgpu fan: 698
Info: Processes: 494 Uptime: 1d 7h 49m Memory: 62.84 GiB used: 7.39 GiB (11.8%) Shell: bash inxi: 3.0.34

Revision history for this message
Ahmet Öztürk (dmxe) said :
#6

OK. I think I have nailed down the problem. The culprit is Glib::ustring::at() function which, for some reason runs rather slowly.
I fixed the problem by switching to std::string and got very impressive result with entries with up to 10K words.

The fix is in master as well as in a newly created 1.5 branch. If you could download the latest version of either of these repositories and confirm that the fix works for you, it would be great.

I recommend going with 1.5 branch as master is in transition to 1.6 version and is not ready for daily usage:
https://code.launchpad.net/~lifeograph-core/lifeograph/+git/lifeograph/+ref/1.5

Revision history for this message
Rick Morris (mushroomancer) said :
#7

Could you push the update to Fedora Testing? I can update the package through sudo dnf upgrade --advisory=<id>

Revision history for this message
Ahmet Öztürk (dmxe) said :
#8

I do not know about Fedora Testing but normally this kind of experimental patches are not uploaded to distribution repositories. I highly recommend building from source.

Revision history for this message
Rick Morris (mushroomancer) said :
#9

Hm, unless I completely misunderstand the purpose of Fedora Testing, it's supposed to be where experimental patches go to die. If a patch doesn't do what it is supposed to do, one simply vote it negative and it will never move to the update pending pile and remain in the "updates in testing" pile to expire, which mean it will never pollute the "updates in stable" pile. I could build from source, but then my testing of it may not be thorough and other bugs might creep in from the change that I may not experience.

Can you help with this problem?

Provide an answer of your own, or ask Rick Morris for more information if necessary.

To post a message you must log in.