this post was submitted on 10 Dec 2023
229 points (96.7% liked)
Technology
58138 readers
4899 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Yep, I spent a month refactoring a few thousand lines of code using GPT4 and I felt like I was working with the best senior developer with infinite patience and availability.
I could vaguely describe what I was after and it would identify the established programming patterns and provide examples based on all the code snippets I fed it. It was amazing and a little terrifying what an LLM is capable of. It didn’t write the code for me but it increased my productivity 2 fold... I’m a developer now a getting rusty being 5 years into management rather than delivering functional code, so just having that copilot was invaluable.
Then one day it just stopped. It lost all context for my project. I asked what it thought what we were working on and it replied with something to do with TCP relays instead of my little Lua pet project dealing with music sequencing and MIDI processing… not even close to the fucking ballpark’s overflow lot.
It’s like my trusty senior developer got smashed in the head with a brick. And as described, would just give me nonsense hand wavy answers.
Was this around the time right after "custom GPTs" was introduced? I've seen posts since basically the beginning of ChatGPT claming it got stupid and thinking it was just confirmation bias. But somewhere around that point I felt a shift myself in GPT4:s ability to program; where it before found clever solutions to difficult problems, it now often struggles with basics.
Maybe they're crippling it so when GPT5 releases it looks better. Like Apple did with cpu throttling of older iphones
They probably have to scale down the resources used for each query as they can't scale up their infrastructure to handle the load.
This is my guess as well. They have been limiting new signups for the paid service for a long time, which must mean they are overloaded; and then it makes a lot of sense to just degrade the quality of GPT-4 so they can serve all paying users. I just wish there was a way to know the "quality level" the service is operating at.
This is most likely the answer. Management saw the revenue and cost and said, “whoa! Turn all that unnecessary stuff off!”