Oh my god did you guys just steal my music?! You each owe me $0.003 !!!
I got half a quesadilla and a shiny rock
You must be a very generous person, that’s worth way more than what dude asked for
You barter with what you have
Let’s put it all on a Funkwhale server.
Sure, you set it up.
Dang. You called me out on my bullshit.
Fuck… Now that RAM prices are skyrocketing, we gonna see hoarders buy hundreds of TB of storage, leading to price hikes
Did not see this coming when I built my 40TB NAS
It would be awesome if we had an app that allowed to stream directly from such torrents, and had a user-made recommendation system to replace the discovery algorithm :D
Stremio + Torrentio does this for TV but I haven’t found an equivalent for music. Hoping to be proven wrong 🤞
I wonder how they are splitting it up in different torrents and how many.
They are splitting it up right? 😁
Has anyone tried to self host this? Of course, hosting 300tb isn’t practical, so any solution would need to download the metadata and songs on demand.
Wasn’t all of this shit already available as torrents?
You could hand pluck grains of rice out of a field or just steal a 25lb bag.
Which would you prefer?
I have 3gb of space to share. Are there 150 other people like me that want to do some distributed hosting? What technology can handle this?
Torrents manage this without any extra tech. Just grab the .torrent file, and only select the files you have space for. Download them then seed, and get your 150 friends to do different files. If I then go to download a file that’s in your batch, the download will happen from your server (and whoever else is seeding these files), and if I go to download a file that’s on one of your 150 others’ server it’ll download from there.
Yes, almost. But i want it to seem like i have all 400tb on my 3tb drive. I want the cache to handle downloads automatically and I want an even spread of files so that none get lost.
Torrents give me:
- Chunking
- Redundancy via multiple seeders
- Partial downloads
But they do not give me:
- A unified filesystem view
- Automatic caching & eviction
Guarantees that every file stays available- User side load balancing or placement control
A way to say “this file must exist on N peers”
Maybe I should be typing this into a LLM.
Edit: ChatGPT is suggesting an IPFS Cluster
Edit 2: If the torrents stays active I can remove some requirements
As far as I’ve read, the database is largely low bitrate files, and some AI. The value here is metadata and preservation of “rare” music.
It’s not lossless but current ogg vorbis at 160kbps is absolutely transparent for the vast majority of people. That’s actually what I chose to keep my own collection, I mean, outside of the lossless albums that I absolutely want to flawlessly preserve.
How many full seeds are there? I mean how many could there be? Who has 300 Tb to throw at this?
Given some of the collections I’ve seen on private trackers I’d say there is going to be quite a few seeding this in its entirety.
Is this new? Aren’t most tracks already available in torrents?
It’s new insofar as this is one big scrape. About 300TB iirc.
300tb is a lot, but its kind of crazy to think this entire company only needs 300tb storage arrays to function. I wonder how they handle things internally. I would imagine at least 1 backup server ready to go in HA. I wonder if they have multiple regions across the country that also serves up the same setup.
Afaik 300 TB is just the most popular music and around a third of all tracks. The blog post on anna’s is quite entertaining tho.
They need other 300TB to store all the ads.
Likely cloned Netflix’s “netflix in a box” design, where they drop a large 200TB+ NAS in thousands of different CDN datecenters with their most popular content cached so that total traffic is minimal across the internet at large.
Spotify mainly being music with very little video likely makes this even easier.
IIRC there’s still like 700TB of low popularity music missing, but it is only something like 0.4% of listens.
And they need a more storage overall because they have to set up datecenters around the world - doesn’t make sense to stream tens of millions of connections across the ocean. But that also gives all the backups one would need for “free”.
That’s disgusting. Where would you find such torrents?
The album art torrent is a goldmine. Such a pain in the ass sometimes to find high quality album covers.
Have they actually been indexed?
There is 200gb of just metadata
4TB if you include all with popularity=0 iirc
Shit me, 200gb in metadata, 2.2TB in cover art before we even touch a piece of music! Wild
Its also crazy when you realise the amount of knowledge an experienced data analyst could gain from 200gb of metadata.
“address not found”
Get better DNS
Nah, checked Wikipedia. Germany is one of the countries that entirely blocks the site.
Luckily I live in Denmark, and Denmark doesn’t block the archive. So when I’m back home I’ll certainly be checking it out.
Not all of Germany. There is this one for example, works like a charm:
People are saying it’s 300TB but this link is only 200GB why?
The 200GB is the metadata sqlite database only
Not released yet
