this post was submitted on 11 Jul 2023
194 points (98.5% liked)

Lemmy.world Support

3222 readers
78 users here now

Lemmy.world Support

Welcome to the official Lemmy.world Support community! Post your issues or questions about Lemmy.world here.

This community is for issues related to the Lemmy World instance only. For Lemmy software requests or bug reports, please go to the Lemmy github page.

This community is subject to the rules defined here for lemmy.world.

To open a support ticket Static Badge


You can also DM https://lemmy.world/u/lwreport or email report@lemmy.world (PGP Supported) if you need to reach our directly to the admin team.


Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world



founded 1 year ago
MODERATORS
 

I am currently getting signed out every minute from lemmy.world. This is not a client side cache issue. I tested making API calls from the command line (with curl) with no cache and the issue still occurs. One call I get the correct response, the next I get a 400 telling me im not signed in.

I'm primarily testing with the https://lemmy.world/api/v3/user/unread_count api endpoint. I'm not sure if this issue occurs with all endpoints.

Reproduction steps:

  1. Get a lemmy.world JWT token for your account using your desired method (eg. postman).
  2. curl https://lemmy.world/api/v3/user/unread_count?auth={JWT_TOKEN_HERE}
  3. Note the 400 error. If you do not get an error repeat step 2.

Edit

This issue only seems to affect lemmy.world so a temporary workaround is to use a different instance for the time being.

top 50 comments
sorted by: hot top controversial new old
[–] antik@lemmy.world 33 points 1 year ago* (last edited 1 year ago) (2 children)

Just a quick statement from the admins team to say that we are aware of the issue and yes we are looking into this.

Thank you @idunnololz@lemmy.world for the elaborate report and everyone else for their patience while we try to sort this one out!

Edit: Lemmy was upgraded to 0.18.2

[–] nosut@lemmy.world 7 points 1 year ago

Thank you for making a statement about it!

[–] beefbaby182@lemmy.world 7 points 1 year ago (1 children)

Thank you for all that you do for this place. I am consistently amazed at how quickly y'all are able to resolve issues.

[–] Spaltovic@lemmy.world 32 points 1 year ago (1 children)

Sounds like lemmy.world runs on 2 instances and the requests are being loadbalanced between those two. That and that the jwt secret is different between those two instances causing one to accept and the other to reject

[–] Zephyrix@lemmy.world 17 points 1 year ago (1 children)

This is also my theory. I think you’re right on the money here. They probably rotated secrets from yesterday’s hack and forgot to restart both servers.

[–] Mereo@lemmy.ca 4 points 1 year ago (1 children)

Does anyone know who can contact the server admins?

[–] Fuck_u_spez_@lemmy.world 7 points 1 year ago (1 children)

Yell real loud in all caps

[–] Laticauda@lemmy.world 27 points 1 year ago (1 children)

Same problem for me it seems, dunno if I'll even be able to comment. Refuses to stay logged in.

[–] idunnololz_test@lemmy.ml 11 points 1 year ago* (last edited 1 year ago) (3 children)

From my tests, it's almost perfectly a 50/50 whether any API requests you make will yield a 200 (success) or a 400 (not signed in). If you perform an action that takes 3 API requests, your chances of succeeding is (1/2)^3 or 1/8 because only 1 request needs to fail in the chain for the entire action to fail. So, as long as you make single API actions you can maximize your success rate :D

[–] marmarama@lemmy.world 5 points 1 year ago

Smells like two instances behind the load balancer, one is fine with the JWT, one is not.

[–] Laticauda@lemmy.world 5 points 1 year ago

Seems like spamming actions also gets it to work eventually. It's a pain in the arse though lol. I made some alt accounts on other instances, but I'm lazy and don't wanna rebuild my subscription feed if I don't have to, so hopefully it gets fixed at some point.

[–] jennwiththesea@lemmy.world 5 points 1 year ago (1 children)

What's an example of something that would take more than one API request?

[–] idunnololz_test@lemmy.ml 4 points 1 year ago

Signing in. Most websites/apps will probably also grab your unread count, and maybe even your subscription feeds.

Another example is checking your inbox. Lemmy actually has 3 inboxes: mentions, replies and PMs. A lot of websites/apps bundle these three so they will need to check all 3 inboxes via 3 API calls.

[–] Metal_Zealot@lemmy.world 14 points 1 year ago

Thaaats what's going on

[–] Mereo@lemmy.ca 10 points 1 year ago (2 children)

Yeah. Lemmy.world is currently unusable on the desktop. I don't have that problem in Memmy. Growing pains but I hope the problem will be fixed soon. Do anyone know if one of the mods in North America are aware of the problem?

[–] Resolved3874@lemdro.id 4 points 1 year ago (2 children)

I was having trouble in liftoff and the browser. Cleared data and cache from liftoff thinking maybe something got messed up there and now I can't even log back into my .world account πŸ€·β€β™‚οΈ I'll hang here for a bit I guess.

[–] trouser_mouse@lemmy.ml 3 points 1 year ago

Same here, can't log in again via Liftoff.

load more comments (1 replies)
load more comments (1 replies)
[–] dnvtr@lemm.ee 10 points 1 year ago

Same issue here, I'm being automatically logged out of my lemmy.world account in Firefox. If I refresh the page even immediately after logging in, I'm automatically logged out.

[–] Sir_Digby@kbin.social 9 points 1 year ago (1 children)

I’m having to reauthenticate in safari and wefwef every time I load a new page. Furthermore, the login is frequently failing.

[–] idunnololz_test@lemmy.ml 11 points 1 year ago* (last edited 1 year ago) (1 children)

Login in likely always succeeding. The issue is that whatever app/website you use will make additional API calls afterwards (eg. fetch posts or fetch unread count). Each of those calls have a 1-in-2 chance to succeed and if any of them fail, they all fail and you will be booted out.

Lemmy is now an RNG game. We must prayge to rngesus before making any actions.

load more comments (1 replies)
[–] AFKBRBChocolate@lemmy.ca 8 points 1 year ago

Okay, so how do we get this fixed? Any way to get admin attention? I think @Spaltovic@lemmy.world is probably correct about the cause.

[–] DelvianSeek@lemmy.world 7 points 1 year ago

FWIW, I can confirm I'm having this issue as well. The load balancing hypothesis seems sound given the behavior I'm seeing. Definitely making lemmy.world pretty much unusable at this point.

[–] krayj@lemmy.world 6 points 1 year ago

I've been experiencing something similar/related. If I am logged in and open something in a new browser window, it frequently (starting today) shows me as not logged in. If I refresh the page, I'm suddenly logged in. This doesn't feel like a authentication problem as much as a timing issue while loading the page. Or maybe what I'm seeing is an entirely different issue.

[–] AlmightySnoo@lemmy.world 5 points 1 year ago

Same here, I thought I was going crazy

[–] TheGoldenGod@lemmy.world 5 points 1 year ago (1 children)

Same here, it’s driving me mad. Also did the above and glad to see I’m not alone!

[–] idunnololz_test@lemmy.ml 2 points 1 year ago (1 children)

The good news is it only appears to affect lemmy.world. If you have an account on another instance, you should switch to that account for now.

load more comments (1 replies)
[–] sheodox@lemmy.ml 5 points 1 year ago

At least when you can't log in on one instance you can just login on another. Downtime doesn't mean you have to go do something else anymore!

I'm seeing the same issues on my app, calling login, then immediately using that jwt to fetch the site details and it doesn't give my_user half of the time, and if my app loads far enough to check the unread count I get not_logged_in

[–] trouser_mouse@lemmy.ml 5 points 1 year ago

Yes I have the issue on Liftoff - can't even log in again. Alt time for now!

[–] Kolanaki@yiffit.net 5 points 1 year ago* (last edited 1 year ago) (1 children)

I can't seem to comment on a couple specific posts on the instance. But as you can see, it works on this one. I am wondering if that's related? I'm not even on my Lemmy.World account and get an unable to post error as soon as I hit the button, like it's not even trying to do anything.

[–] idunnololz_test@lemmy.ml 4 points 1 year ago

From my experience it's entirely random. You can make 5 actions and all 5 will work. Then have a string of 5 actions where none would work.

[–] Bunnylux@lemmy.world 4 points 1 year ago

This is happening in the connect app and I thought I was going crazy.

[–] roon@lemmy.ml 4 points 1 year ago (1 children)

Lucky I have a .ml account

[–] ikantolol@kbin.social 4 points 1 year ago (4 children)

is there a way to see lemmy instances sorted by subscriber amount ? I might wanna join the smaller ones lol

[–] tj111@lemmy.fmhy.ml 3 points 1 year ago* (last edited 1 year ago)

Lemmy.Fmhy.ml is a sweet spot. It's in the top 10, but further down the list. Active and ran by cool admins. No issues from massive user counts (so far). The community is pretty strong and active so i imagine it's got some staying power.

[–] harry_assman@lemmy.world 2 points 1 year ago

https://lemmyverse.net

It letβ€˜s you sort for all kind of things in regards of instances and communities.

load more comments (2 replies)
[–] kwot@lemmy.world 4 points 1 year ago

Yeah, was wondering what was going on. Hope it gets fixed soon!

[–] fubo@lemmy.world 4 points 1 year ago

I'm getting this too, even after clearing cookies and logging in again. I've seen it on multiple devices (Android phone, Linux desktop with Chrome).

[–] melonpunk@lemmy.world 4 points 1 year ago

Making a new post is a nightmare. I wish the submit button would time out in these instances so you can try again. Right now I'm having to copy and paste into a new tab and hope for the best (but fail, 5 times and counting).

[–] kescusay@lemmy.world 4 points 1 year ago

Just add me to the list. Jerboa seems to stay logged in for about 75% of interactions.

[–] xaon_rider92@lemmy.world 3 points 1 year ago

I'm having the same issue... might switch to an alt or something for now.

Might be worth noting that the Google password manager doesn't recognize to auto fill

[–] ElectroVagrant@lemmy.world 2 points 1 year ago

I noticed it a little yesterday, but strangely it seems worse today. I'd cleared cache & cookies after the compromise issue, so I was pretty sure it wasn't to do with not doing that, however I think OP's on the right track with their assessment.

Not really sure why it would suddenly seem worse though, other than maybe something to do with server traffic/activity.

[–] harry_assman@lemmy.world 2 points 1 year ago (3 children)

Same here. All Edge, Firefox and Safari keep logging me off. Same with Memmy iOS app.

load more comments (3 replies)
load more comments
view more: next β€Ί