this post was submitted on 05 Feb 2025
27 points (86.5% liked)

Fediverse

18142 readers
151 users here now

A community dedicated to fediverse news and discussion.

Fediverse is a portmanteau of "federation" and "universe".

Getting started on Fediverse;

founded 5 years ago
MODERATORS
deadsuperhero@lemmy.ml
wakest@lemmy.ml
27
Question/Suggestion: A Fediverse Equivalent to the Internet Archive? (lemmy.world)
submitted 1 day ago by Teknevra@lemmy.world to c/fediverse@lemmy.ml
9 comments fedilink hide all child comments
 

Hello everyone,

Has anyone ever considered potentially building or launching a platform similar to the Internet Archive but using ActivityPub?

This could serve as a decentralized network to document, preserve, and protect online content from loss, censorship, and other threats, ensuring its availability for future generations.

For those unfamiliar, the Internet Archive is a non-profit that has been preserving digital media and promoting universal access to knowledge since 1996.

It’s famous for services like the Wayback Machine and Archive-It.

Given the importance of preserving digital heritage, especially in the context of censorship and data loss, a Fediverse-based equivalent could fill a crucial role.

The decentralized nature of ActivityPub could provide a robust alternative to centralized solutions.

I'd love to see this kind of project come to life, but, unfortunately, I lack the motivation, time, and energy to take it on alone.

Has anyone else ever considered something similar?

Are there any existing projects that might be interested in this direction?

Internet Archive Wikipedia

you are viewing a single comment's thread
view the rest of the comments
[–] AbouBenAdhem@lemmy.world 7 points 1 day ago* (last edited 1 day ago) (2 children)

The issue I see is ensuring that a distributed archive is comprehensive. How do you know what’s missing and needs to be added unless there’s a central coordinating process aware of what everyone already has?

[–] grue@lemmy.world 6 points 1 day ago (1 children)

Internet Archive itself has apparently been involved with something called Filecoin, which I assume would solve that kind of issue with 'blockchain,' somehow.

https://blog.archive.org/2023/10/20/celebrating-1-petabyte-on-the-filecoin-network/

[–] floofloof@lemmy.ca 4 points 1 day ago (1 children)

Is that like the usual blockchains where every computer has to store a complete copy? That would get huge with the Internet Archive.

[–] asudox@lemmy.asudox.dev 4 points 1 day ago* (last edited 1 day ago)

No, just some metadata:

Filecoin is an open protocol and uses a blockchain to record participation in the network.

[–] floofloof@lemmy.ca 4 points 1 day ago* (last edited 1 day ago)

There are distributed filesystems with redundancy, but the last time I tried something like that, it was extremely slow for both reading and writing. For an offline archive it might be feasible, but you'd have to do a lot of redundancy and error correction to be sure you didn't lose chunks. Plus, the Internet Archive is so big that even with the data distributed, each participant might have to store a prohibitively large amount.