this post was submitted on 23 Jun 2023
17 points (94.7% liked)

Experienced Devs

3956 readers
1 users here now

A community for discussion amongst professional software developers.

Posts should be relevant to those well into their careers.

For those looking to break into the industry, are hustling for their first job, or have just started their career and are looking for advice, check out:

founded 1 year ago
MODERATORS
snowe@programming.dev
jmk1ng@programming.dev
drewsiferr@programming.dev
17
What's the most cost-effective way to serve lots of large files (lemmy.ml)
submitted 1 year ago by omegastick@lemmy.ml to c/experienced_devs@programming.dev
15 comments fedilink hide all child comments
 

I'm looking at starting a service that involves hosting a lot of LLM models, which are often going to be 16GB+ (compressed). I did a bit of searching for cloud storage providers with cheap egress, and the cheapest I could find is $0.01 per GB, which would still be $0.16+ per download.

How do sites like Huggingface or CivitAI do it? Lots of VC funding?

you are viewing a single comment's thread
view the rest of the comments
[–] minorninth@lemmy.world 1 points 1 year ago (1 children)

That's true, but I just checked a few CDNs and you won't find one for less than $.01/GB. The lowest I found was $.03/GB.

[–] otl@lemmy.sdf.org 4 points 1 year ago* (last edited 1 year ago) (1 children)

To keep costs down and depending on how much you want to get your hands dirty, you could start investigating renting dedicated servers. Some hosting providers offer unmetered network connectivity. Here's something from OVH: https://www.ovhcloud.com/en/bare-metal/rise/rise-stor-1/

And hey, depending on how grassroots the project is, there's always bittorrent! ;)

[–] omegastick@lemmy.ml 3 points 1 year ago

I was considering this. The hosting provider we use for model training runs doesn't charge for ingress/egress. Their storage costs would eat us alive though haha. OVH looks much more promising.