this post was submitted on 11 Jun 2024
94 points (100.0% liked)

technology

23182 readers
458 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 4 years ago
MODERATORS
 

The big AI models are running out of training data (and it turns out most of the training data was produced by fools and the intentionally obtuse), so this might mark the end of rapid model advancement

you are viewing a single comment's thread
view the rest of the comments
[โ€“] QuillcrestFalconer@hexbear.net 24 points 3 months ago (1 children)

Eventually researchers are going to realize (if they haven't already) that there's massive amounts of untapped Data being unrecorded in virtual experiences.

They already have. A lot of robots are already training using simulated environments, and nvidia is developing frameworks to help accelerate this. Also this is how things like alpha go were trained, with self-play, and these reinforcement learning algorithms will probably be extended for LLMs.

Also like you said there's a lot of still untapped data in audio / video and that's starting to be incorporated into the models.

[โ€“] JoeByeThen@hexbear.net 16 points 3 months ago

Yeah, I'm familiar with a bunch of autonomous vehicles/drones being trained in simulated environments, but I'm also thinking stuff like VRChat.