this post was submitted on 30 Jun 2023
383 points (93.8% liked)

Lemmy.World Announcements

29079 readers
274 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
 

We'll give the upgrade new try tomorrow. I've had some good input from admins of other instances, which are also gonna help troubleshoot during/after the upgrade.

Also there are newer RC versions with fixed issues.

Be aware that might we need to rollback again, posts posted between the upgrade and the rollback will be lost.

We see a huge rise in new user signups (duh.. it's July 1st) which also stresses the server. Let's hope the improvements in 0.18.1 will also help with that.

top 50 comments
sorted by: hot top controversial new old
[–] 00Lemming@lemmy.world 102 points 1 year ago* (last edited 1 year ago) (9 children)

PSA from Admin Team: The update completed roughly two hours ago. Since that time, the Admin team (and other site admins) have been working on the noted performance issues. We believe we have found a solution, but we still need time to test this out. You may still see brief outages and differences in performance as we are testing different configurations. We are trying to prevent rolling back.

While I know this can be frustrating - especially today - please keep in mind we have a team of volunteer techies (from around the globe!) collaborating on this issue. It is an inspiring situation. Also keep in mind that lemmy.world is quite a bit larger (and more active than any other instance). As such, we are a bit of a β€˜test instance’ in regards to high volume requests. This is just part of the growing pain. We appreciate your understanding.

@ruud@lemmy.world will provide a debrief once we have completed testing.

[–] TurnItOff_OnAgain@lemmy.world 25 points 1 year ago (1 children)

We are happy to have all of you! Do what is needed to make this place the best. Even reddit sucked in the early days.

502 it went through, 504 try once more

[–] 00Lemming@lemmy.world 8 points 1 year ago (1 children)

Thanks for the kind words! Yeah, there are definite growing pains, and likely will be for some time (just do to the codebase we are working with, understandably). We have a really solid group though heading up lemmy.world though so we will be just fine ;)

[–] illi@lemmy.world 7 points 1 year ago (1 children)

Not sure if this is a good place to ask but it was on my mind these days, with the big user boom and probably most of it being directed at .world - should some of us consider moving to other instance to make room? Would it make sense? Would it help? Or are you ok having so many users under the .world umbrella (possibly causing even greater flood with people seeing it is the "main" (with biig quotes there) instance?

[–] 00Lemming@lemmy.world 8 points 1 year ago* (last edited 1 year ago) (3 children)

Technically speaking, yes, a portion of our issues are due to the highest user base of an Lemmy instance. So in theory, if half of our users dispersed to other instances, we would likely see some performance improvement here. However, lemmy.world is intended to be an accessible instance for the general population. The server itself that is running lemmy.world is beyond spec'd to handle much more than this user load. We are running up against code-level issues that we may or may not be able to get around with our internal configurations. This is just part of developing software in an environment were you go from a few thousand users total to hundreds of thousands in the space of a few weeks. There is no directive to have users create accounts on new instances, though if you are looking for an immediate performance improvement, that may be your best option currently. That is up to you to decide :)

load more comments (3 replies)
load more comments (8 replies)
[–] simple@lemmy.world 60 points 1 year ago* (last edited 1 year ago) (3 children)

Server feels a lot better now than it did an hour ago. Comments are going through quickly and upvotes are working for me.

[–] ekZepp@lemmy.world 25 points 1 year ago* (last edited 1 year ago) (1 children)

Upvote are still slow. I'm testing the comment right now.

Ok, commented are fine ( edit too) πŸ‘

[–] TheWorstNL@lemmy.world 8 points 1 year ago

Upvotes are still slow indeed but they are at least appearing now after 15 seconds. πŸ‘

load more comments (2 replies)
[–] rtxn@lemmy.world 23 points 1 year ago* (last edited 1 year ago) (1 children)

Performance is extremely uneven. Sometimes loads instantly, sometimes I get a timeout. Upvotes don't show up until reload. Still a lot better than the last attempt.

The server hamsters seem to be on fire, but I won't hold it against the service/community/individuals (except spez; fuck you spez). Lemmy is in the unfortunate position of being forced into a development model called "fuck it, we'll do it live!"

edit: commenting and editing seems to be fine.

[–] skipseagull@lemmy.world 5 points 1 year ago

Seems to be evening out

[–] MrCenny@lemmy.world 20 points 1 year ago (10 children)

Heyo! Small update from someone who is watching the upgrade live; it's ongoing still. Seems like they are still facing some performance issues. So grab your popcorn and wait! 🍿 (Btw, i am not a sysadmin, just a moderator)

load more comments (10 replies)
[–] Anaralah_Belore223@lemmy.world 20 points 1 year ago (1 children)
load more comments (1 replies)
[–] trachemys@lemmy.world 19 points 1 year ago (3 children)
load more comments (3 replies)
[–] Spectator@lemmy.world 15 points 1 year ago (1 children)

And… we’re back!

[–] IndependentRanger@lemmy.world 15 points 1 year ago (2 children)
[–] Barns@lemmy.world 9 points 1 year ago

Took a sec but upvoting works πŸ’ͺ

load more comments (1 replies)
[–] riodoro1@lemmy.world 12 points 1 year ago (8 children)

TFW you run a small server with an open source social platform for shits any giggles and one month it explodes a thousandfold because of one greedy pig boy.

load more comments (8 replies)
[–] WantsToPetYourKitty@lemmy.world 11 points 1 year ago (2 children)

Nice work with everything so far, really liking it. What all is being changed/fixed in the new revision? Is it just more efficient code to support higher site traffic?

[–] TeaHands@lemmy.world 6 points 1 year ago

Biggest thing is it entirely gets rid of the problem where the All feed would auto-update and add a ton of random posts while you were trying to browse. Really awful experience browsing by All, for a while there.

load more comments (1 replies)
[–] Quetzacoatl@lemmy.world 10 points 1 year ago

looking good so far, the instance is back, testing to post!

[–] MrNemobody@lemmy.world 10 points 1 year ago (4 children)

I can't login on my lemmy.world account, neither on Jerboa nor in the browser, I'm getting "Incorrect login" :(

(I'm posting from Liftoff because I was already logged in)

load more comments (4 replies)
[–] teft@lemmy.world 9 points 1 year ago
[–] nexguy@lemmy.world 8 points 1 year ago (4 children)

Posting and seems fine from Jerboa, can't log in using wefwef.

load more comments (4 replies)
[–] AllGoesUpMustGoDown@lemmy.world 8 points 1 year ago* (last edited 1 year ago)

Good luck! Posted 19:59 CET.

[–] josephos@lemmy.world 8 points 1 year ago (2 children)

Looks like it works, but it's very slow - 25 seconds to load the page. https://i.imgur.com/ghiofDC.png

load more comments (2 replies)
[–] randommodule@lemmy.world 8 points 1 year ago

Seems to have worked! Typed from Jeboa :)

[–] dragontamer@lemmy.world 7 points 1 year ago (1 children)

Server has sped up significantly. Things are getting faster / better.

Maybe 0.18.1-RC4 will stick? Lets hope!

load more comments (1 replies)
[–] ekZepp@lemmy.world 7 points 1 year ago (1 children)

Is the update done? I hope not because can't i upvoteπŸ˜…

load more comments (1 replies)
[–] Anaralah_Belore223@lemmy.world 7 points 1 year ago* (last edited 1 year ago) (1 children)

Look like Lemmy.world got slightly better.

This is a test comment. Commenting took about 5 seconds

EDIT: Test edit took about 3 to 5 seconds

load more comments (1 replies)
[–] paorzz@lemmy.world 7 points 1 year ago (2 children)
load more comments (2 replies)
[–] AlmightySnoo@lemmy.world 7 points 1 year ago
[–] Nuggets@lemmy.world 6 points 1 year ago

Finally able to register. Glad to be here.

[–] IndependentRanger@lemmy.world 6 points 1 year ago (1 children)

Feels very sluggish for me

[–] TheWorstNL@lemmy.world 6 points 1 year ago

I think it's still struggling with the high user influx from Reddit and the 3rd party apps which stopped working today. And probably after the upgrade caches have to be rebuilt. Hope it improves soon.

[–] savvywolf@pawb.social 6 points 1 year ago

Godspeed you absolute madlads.

[–] AlmightySnoo@lemmy.world 6 points 1 year ago (5 children)

Cleared my cache and cookies as I thought it would be necessary like last time, but now I can't log in. The login button just turns into the animated wheel and then nothing happens. I'm typing this on a mobile where I still have an active session.

load more comments (5 replies)
[–] Grangle1@lemm.ee 6 points 1 year ago (1 children)

Just a heads-up. I ended up having to create a new login at lemm.ee because even after the improvements in speed and so on with the upgrade, for all intents and purposes it looks like my account here may have been wiped. Can't login from anywhere, getting "Record not found" whenever I try to sign in on Connect, "incorrect login" from Jerboa, and a spinning button and then nothing on a PC browser, basically treating my account as if it doesn't exist. Some folks appear to be able to login though. Is this an ongoing server issue, or did my account actually get wiped? Same username (Grangle1) as on lemm.ee.

[–] pampoon@lemmy.world 8 points 1 year ago

Logins are returning non found errors. It’s a bug. If you had a previous session that’s as already logged in, you should be able to access it.

[–] Jordynl@lemmy.world 5 points 1 year ago (3 children)

Can't you disable the post button within the upgrade progress? It's a shame to see posts to disappear due to it.

load more comments (3 replies)
[–] teft@lemmy.world 5 points 1 year ago

Looks like it worked! Thanks for all you do @Ruud!

[–] Sterben@lemmy.world 5 points 1 year ago

Thank you for the heads-up. πŸ€™

[–] simple@lemmy.world 4 points 1 year ago* (last edited 1 year ago)

Success?! Hello from 0.18 :)

The UI is a lot smoother. Feels good man.

Edit: servers are very sluggish though. Hopefully this gets fixed without needing to roll back.

[–] BitOneZero@lemmy.world 4 points 1 year ago* (last edited 1 year ago) (1 children)

My lemmy server is also running the same versions are you are now. I was getting timeouts with lemmy.world federation so I recompiled lemmy_server with timeout changes:

diff --git a/crates/utils/src/lib.rs b/crates/utils/src/lib.rs
index e5d07db2c..e2c592d82 100644
***
a/crates/utils/src/lib.rs
+++ b/crates/utils/src/lib.rs
@@ -18,7 +18,7 @@ use std::time::Duration;

 pub type ConnectionId = usize;

-pub const REQWEST_TIMEOUT: Duration = Duration::from_secs(10);
+pub const REQWEST_TIMEOUT: Duration = Duration::from_secs(13);

 #[macro_export]
 macro_rules! location_info {
diff --git a/src/lib.rs b/src/lib.rs
index cc77ca48f..45c621a7c 100644
***
a/src/lib.rs
+++ b/src/lib.rs
@@ -37,7 +37,7 @@ use tracing_subscriber::{filter::Targets, layer::SubscriberExt, Layer, Registry}
 use url::Url;

 /// Max timeout for http requests
-pub(crate) const REQWEST_TIMEOUT: Duration = Duration::from_secs(10);
+pub(crate) const REQWEST_TIMEOUT: Duration = Duration::from_secs(16);

 /// Placing the main function in lib.rs allows other crates to import it and embed Lemmy
 pub async fn start_lemmy_server() -> Result<(), LemmyError> {

load more comments (1 replies)
load more comments
view more: next β€Ί