Lemmy Server Performance

3 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 1 year ago

MODERATORS

RoundSparrow@lemmy.ml

RoundSparrow@bulletintree.com

239

GREAT NEWS about Lemmy Server Performance, another major SQL mistake has been discovered today: every single comment & post create (INSERT) is updating ~1700 rows in the site_aggregates table (lemmy.ml)

submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/lemmyperformance@lemmy.ml

32 comments fedilink hide all child comments

Details here: https://github.com/LemmyNet/lemmy/issues/3165

This will VASTLY decrease the server load of I/O for PostgreSQL, as this mistaken code is doing writes of ~1700 rows (each known Lemmy instance in the database) on every single comment & post creation. This creates record-locking issues given it is writes, which are harsh on the system. Once this is fixed, some site operators will be able to downgrade their hardware! ;)

you are viewing a single comment's thread
view the rest of the comments

[–] darth_helmet@sh.itjust.works 11 points 1 year ago

This is such a dumb take. Using a database efficiently is not some binary, once-off thing: you build what works based on the data you have at the time. When it works, you move on to other features. It takes analysis of real operation over time to find the bottlenecks, and discipline to focus on fixing the things that will have the most benefit to your users.

There are many successful tech companies who introduce features that create dogshit performance impacts regularly. They work because there are people looking at metrics and catching issues to fix. This is healthy.