A Cult Kills the-eye.eu: Raising funds for rebirth at 10Gbit/s (and new data of course)
Hey guys, it's been awhile. I'll try to make this a short version of events.
A church/cult (amorc) effectively got our previous host (datapacket) to drop us
We went colo, bought new hardware, have uncontested 10Gbit/s
Our spinning rust can only serve at 1.2Gbit/s
We're looking to raise funds to buy new ssds so we can make use of this 10Gbit/s
Our new server is on a sub domain for those in the know on our discord
Our primary domain points at temp server with around a 3rd of our content
Our current donors have already raised $400 in the last few days, we still need another $700 for these upgrades. To note, all donors that join our discord get access to exclusive releases, first to hear news and behind the scenes info, exclusive giveaways, invites, etc.
Until we raise the funds for new ssds we're operating at a very limited capacity.
Upgrade Options
8TB - PCIe/M.2 based (PCIe 2.0, raid0 5Gbit/s)
$119.90 -
amazon.com/SEDNA-HyoperDuo-acceleration-function-included/dp/B01EHTD0VK$224.99 -
amazon.com/Blue-NAND-2TB-SSD-WDS200T2B0B/dp/B073SBW3VDx4 - $899.96
Total - $1019.86
Pros: Fast AF!! No power mod to server to install
Cons: Raid0 volatility, config/driver
8TB - 2.5" Sata based (4.4Gbit/s at drive, 3Gbit/s on controller)
$547.99 -
amazon.com/Samsung-Inch-SATA-Internal-Gray/dp/B07L3CLM2BTotal - $1095.98
Pros: Common hardware, no config/driver
Cons: Speed limited by sata2, power mod to server and jerry rigging
Seems to make more sense going with pcie for ease, no? What do you guys think?
New Data @ 10Gbit/s
Here I've put some new data on our current ssds in the new server, mostly for testing our new connection.
This is a dump of the Higher Intellect collection @ preterhuman.net and will serve as a 10Gbit/s mirror after the owner rate limited downloads at the source. Read more about this collection here: https://preterhuman.net/about.php
137.4GB in 592,600 items. (ncdu)
Get it fast as possible: curl -s http://beet.the-eye.eu/hi |parallel -j 42 'wget -m -np -c -R "index.html*" {}'
thecoverproject.net Game covers dump...
/site -
60GBin16,582items./raw -
21GBin3451items.
There are many things in here, I haven't sorted past extension. Texts, books, etc, idk.
22GBin61,314items.
42GB of Avant-Garde - Experimental Cinema
Flashpoint 6.0 Ultimate: Everything in one offline download. 102GB
For more info on the Flashpoint project visit: http://bluemaxima.org/flashpoint
vulnhub.com maybe going away so here's 286GB of the VMs and such they host.
magnet:?xt=urn:btih:4522bd9fdd4667dcae3d010c5853f06c88a50cb1&dn=vh
Community
You can reach me here on reddit, in the r/DataHoarder IRC (GreenObsession) or on our discord server. Come chat to everyone, see our new content before anyone else and join other like minds.
Supporting Us
If you like what we do consider donating towards our server costs.
$600.00/month covers all of our service costs.
BTC: 3Mem5B2o3Qd2zAWEthJxUH28f7itbRttxM
Someone should add a "controversy" section to this page https://en.m.wikipedia.org/wiki/Ancient_Mystical_Order_Rosae_Crucis
But why are they intersecting? Why does AMROC care about theeye?
Interestingly enough, the historical Rosicrucians weren't ever supposed to charge for their teachings....I guess things get diluted over time.
Why would a church/cult make your host drop the site? It doesn't make much sense, unless I'm missing something very obvious.
There's info here in our discord #news channel
But basically the church is dying off, they own a lot of real-estate they struggle to afford so they do things like copyright troll for cash, in this case they wanted to sue us for $125,000,000 for hosting 170 PDFs, instead of simply sending a DMCA request like normal people they harassed our host and everyone else involved for months via cease and desist even after the files were removed.
Oh wow. Not really surprised that they are doing this simply to try and get loads of money. Actually sad in my opinion, especially their ridiculous demands as well (like who demands 125 million for less than 200 PDF documents).
I do hope that you're able to get this whole situation sorted out or resolved.
You should make silly communications back n fourth with their lawyers to cost them a bunch more money. Lol
He's about to retire, I assume that this was his final go at a large money grab before he kicks the bucket.
Can this thread be pinned somehow? Otherwise people would just forget about it after a while...
That would be nice, but entirely upto /u/_ze & /u/MrDorkESQ
The-Eyes first home was this sub, but without posting weekly people do forgot we exist, which is sad because we are an open directory built by the community for the community.
Unfortunately, threads can't be pinned unless they are created by a mod or admin.
I could create a new thread and pin that, but it would be a new thread.
That's odd, I can pin anyone's thread over at /r/DataHoarder is that a settings thing? All you have to do is hit make announcement
I've been modding on here for 7 years and never knew that. Yeesh!
/It's not like that is obvious though.
Nah, pretty obscure little button under the post. Took me awhile to figure it out a year or so after they announced the feature haha.
Thank you!!
Idk was thinking it could help with the comeback of The-Eye. Anyway I'll ask a bunch of friends around to see if they could donate. Is there any other hardware you're looking for? I might know some leads...
We've never actively advertised anywhere but here, we've made a few posts to 4chan for our content that fits over there, but no traditional advertisement no.
We're strictly community funded, this sub was the foundation of our community, we want to stick with showcasing to like minds, people who know what to expect from us. When we got the attention of the rom crowd that was the first time we stepped outside of our comfort zone and some say it didn't go well, while it did bring in some new donors general rom users tend to need a lot of hand holding and babying which got old fast as we never intended to be a rom site.
I don't know really, I haven't put much more thought than this into it.
As for hardware we could make use of ram, (dell r710 is our platform) ssds, 8TB+ hdds right now, though generally we will take and make use of any and all free hardware should it make sense to ship it to us (performance vs shipping costs)
"no legacy is so rich as honesty" That lawyer is a prick no matter how he spins it.
Wait wait wait. Did I just read it right. A cult or sect closed down the-eye.eu? Is this a storyline of a cheap tom clancy book?
It is so unreal I just can't believe it.
Anyway, I'll donate via Paypal
Hey I like Tom Clancy books, haha. But you read right, they forced our host to drop us, didn't manage to stop us however, so this post serves as a rebirth of sorts, we're still up just running a little slow and scattered.
One can saturate 10Gbit reads with ZFS on WD datacenter 7200rpm mechanical disks. Obv faster disks would be better, but just sayin.
SSD caching would help lessen I/O, but going full SSD is overkill here IMO with most I/O being reads not writes.
We should keep in mind that the same files aren’t being hit each time, and there are usually 3-7 people wgetting entire collections at any given time. Seek time is probably what’s killing IO most. Flash would help a lot in this respect.
I'm ashamed to admit I am one of those users... sometimes 2 instances concurrent. :(
Also typically me, I've been cloning the entire archive to my home server.... Whoops
They aren't even a real cult, they just placed ads in the back of lots magazines like Omni and Popular Science and got people to sign up for over priced mail order lessons. Other than a so called museum and HQ in San Jose there is nothing else, no chapters, no secret meetings.
They own property all over the world, have members in many countries, but a quick look over their workings these days shows clearly they're slowly but surely dwindling in membership and are doing anything they can to stay relevant and raise funds to pay for the upkeep of their real-estate.
Do they charge money for their info? I haven't had time to look into them but I feel having their info hosted would spread and increase their numbers.
So I've read conflicting information. I think they charge for leveling up to different degrees within the organization.
How much RAM does the box have? It probably makes sense to invest in gobs of RAM before maxing out the storage specs, because Linux efficiently uses RAM for I/o cache.
Additionally, if the filesystem is ZFS, adding a NVe raid0 ARC is a great way to speed up I/o without risking data loss.
/u/-Archivist ^ didn't you say you had more ram to be installed, still at 128G?
Given chat on the discord I think arch wants to do this in place rather than reroll the install to change filesystems etc. He's also got no experience with ZFS on Linux.
Yeah we're still at 128GB ram but do have leads on more for free, I'm yet to chase that up. However given we host 32TB now and our platform maxes out at 768GB ram filling wouldn't help so much.
I've been looking into a few solutions, fs tuning, etc. but simply moving our most popular files to ssds is the best option to get this issue solved now rather than later, it's already been over a month we've been in this position.
I'd recommend looking into the Intel optane persistent memory. If your platform supports it.
Way cheape, and higher capacity, than RAM, and mostly just as fast.
If this is an older server you could source some 16GB DDR3 modules from recycling centers for super cheap.
IT Admin w/ 20+ yrs experience chiming in here. Do not use raid 0.
If you lose even a single disk; that's your array and all the data gone. Depending on the number of disks you're using; you want a minimum of raid 10 and likely raid 60 as to sustain multiple simultaneous drive failures.
I would recommend you look at 1.8-2TB 2.5" SSDs on SATA. Yes you're going to be limited by the SATA bus speed but it's still going to provide more than enough performance to max out 10Gbps when ran in raid 60 or similar.
Feel free to PM me if you want to discuss your needs in more detail; if I had a bit more info on the hardware chassis itself I could prob provide more specific details.
Uptime vs speed. For a non critical site like the eye, uptime doesn't matter nearly as much as speed.
Do not use raid 0.
Well aware of this, however we chose r0 on purpose allowing us to serve more data. Now we're in colo a data restore should a disk have to be replaced would only take around 14 hours, we would rather take this hit than serve less data by going with a redundant system.
We're currently looking at 4TB 2.5" ssds on sata but the spare ports we have are sata2 (onboard the r710 mobo, replacing the dvd drive, this was taken into account if you see what I listed in the post) or nvme on pcie cards as we have spare pcie slots and it works out slightly cheaper than the 4TB sata drives.
But yeah, we're working with a dell r710.
Just to be clear; if you lose one disk - you lose all of the data on the whole array. This would mean rebuilding your whole array; not just the data from the one disk you lost.
In my experience; 14 hours would be wildly optimistic for a restore of that much data. The slow part is typically the backup server/software - it's not going to be maxing out your disks or connections.
Your call in the end; but there's no way I'd build any kind of system without raid.
This would mean rebuilding your whole array
Yup, I know. I'm quite well informed on all of this but thank you for your input :)
I would. JBOD with file checksums and backups, a MAID system for lack of writes, or Spanned storage. Not all of us can afford to disk-match our storage arrays, and since this is a community project who knows what budgets they're working with. Seriously, for your own personal business or for mission critical operations a RAID system is preferred, but not absolutely needed for a project like this. The odds of a bit flip are a probability of 3.7 × 10-9 per byte per month. Or in other words, realllly unlikely. Checksum checks every 30 mins or so can greatly minimise this to near 0.
Now, if you're storing bank transactions or medical records, I'm with you all the way on RAID.
My stars!! There's 0 delay in pageloads!!! :O
What kind of witchcraft is this, and who do I rent my soul to for this power?
Nostalgia kicks in. Does anyone know other dumps of texts on hacking/phreaking/whatnot similar to preterhuman.net? Is this still a thing?
Hi, I'm on mobile and the PayPal link at the bottom is bringing me to the reddit registration page. I just want to give you my money.
The PayPal link is a pre-formatted message to me, we use a few paypal accounts so when I'm messaged about it I give one out for you to send to, I PM'd you.
Because it's pointing at the temp server we have on hetzner to the end of the month before we point the primary domain at the new colo server after the new ssds are installed. All this is explained in our #news channel on discord, we post all server related changes there before reddit.
Have you considered moving to BitTorrent or even BitTorrent Sync to decentralize the database? I have a not insignificant amount of space on a home server that could help distribute items. It's not a massive pipe but if enough people sign up it could greatly help. BitTorrent Sync could even let you change the database on the fly and have it distributed to the other seeds, but it's not as simple to set up as just plain torrent software.
I was waiting for this one, yes we've considered it, no we're not doing it.
This has been suggested no less than a 1000 times in the-eyes life time, there are many many reasons we're not doing it. We will always be open directory as we are now.
How about IPFS network? At a first glance it looks like best of both worlds: it is open and works like p2p network
i once tried talking about using demonsaw routers with these dudes. I was ate alive. Thrown out to dry like i was tonight. Like how can they be so smart and so fucking dumb at the same time i think to myself.
The hardest thing in the world is to protect someone from themselves. Right now they're basically a flashing light for anyone that wants to DMCA them. And from what I've seen they're loaded with DMCA-able content. If I mention TOR, will I be dismissed out of hand like I was with the decentralized options?
If you mention anything about tor they will ban you first chance they get. Happened to me there. The one mod went out of his way saying you can't just have a hidden website. After I stuck my throat out saying how there could be it was silence. But anything i did he was there on me after. I ended up posting a text based meme that must have hit to hard because I he banned me. I was the guy preaching a hidden service along with public hosted demonsaw routers could work. They care nothing about opsec and being hidden. I guess that is their plan though.
I agree. They have an awesome thing going that needs projected. When I first started using the chat there long ago before the great meme ban of yesterday god rest pepes soul I thought they would be all about opsec. Its really just some 14 year olds trying to level up a discord bot. I am salty because these dudes.
/u/Camelfoe Let's cover your suggestions and concerns for the 1000th time, I'll copy/paste the pinned message about it in #suggestions on our discord first.
Hey, we're compliant with dmca and always have been, always will be. C&Ds hmm, those are something else and we just deal with them case by case. Everything you suggested I've been involved in, tor, ipfs, i2p, etc. Sure great technologies, the-eye however will always remain public, open, easy access for all with a simple browser. This has always been our intent, we don't plan to start hiding now.
Futhermore a site of our size couldn't exist on tor, etc. not due to a technological limitations but simply because we're entirely community funded and the community wouldn't fund something with such a high barrier to entry, while me and you may find it easy to route through tor or proxy chains the general user doesn't and/or can't be bothered to, driving them elsewhere.
I've now lost count of how many people have suggested what you're suggesting now, or how many times I've told people to go away, this is the first time I've made effort to cover exactly why we're not going down that road, so see yourself lucky. Myself and my staff are all well aware of or have used the technologies you're suggesting, read: we're not fucking stupid. also read: we exist as we are on purpose, fuck.
So, you having read that it should cover everything, but for good measure lets talk about torrents. We don't offer many torrents due to the very large file count of the datasets we host, we want to remain indexed by google etc so people can drive by download individual files, creating torrents of most of our datasets would mean packing the sets into zips or equivalent to not cause torrent compatibility issues with many common clients. I feel like I constantly have to state my credentials in running the-eye, I've been involved data distribution via many means over that last 18 years or so having run my own private trackers, ftp trading sites, DC++, etc.
The-Eye exists as an open directory on purpose, I'm well aware of all the other possible methods of distribution and I've used them myself, again the-eye is an open directory and dmca compliant on purpose, I've covered this so many times it hurts to be talking about it again. People like you two seem to talk down to me like I'm unaware of the technologies you're suggesting or just simply dumb. I can't stress enough we operate exactly as intended and we're not just stumbling around with our thumbs up our assholes making it up as we go along.
Its really just some 14 year olds trying to level up a discord bot.
I don't know what this is in reference to honestly, but I think anyone reading this would call you out as the 14 year old. Among other things you were banned because you posted a meme about school shootings, that's just in very bad taste and unfitting for our discord no matter which way you look at it.
I thought they would be all about opsec
Why would you think that? We're very open about what we do and how we do it, we don't however expect people to conduct themselves so poorly and attack us over silly and childish things such as memes, no matter the content of the meme. To lower myself to your level, you seem to think the internet is serious business.
but until they start to take security seriously I'm afraid it would just be wasted.
Again, I'm unsure what you mean by this, do you expect us to be a hidden service on bulletproof servers in order to support what we're doing? If so, you're effectively asking us to serve less data to less people, a 32TB site on the tor network wouldn't seem more than a terabyte of traffic a month and even that is a high estimate, currently we're serving in excess of one petabyte each month and that's impossible via tor, etc.
Also we're community funded, the barrier to entry tor adds would mean we lose 1000's of users overnight, you and myself may know how to access hidden services and it's by no means hard but the general public have no idea what tor is and simply don't care to learn.
Right now they're basically a flashing light for anyone that wants to DMCA them. And from what I've seen they're loaded with DMCA-able content.
On purpose, for many reasons we can discuss if you want, but this comment is already long enough. I hope the three of us can remain civil and continue to discuss what I've addressed here but I think the bottom line is that because we're not running the site exactly how you want you're just not happy and I have to say as two people who talk a lot of smack but haven't put your money where your mouth is I think you just enjoy complaining.
I think you misunderstand me. I love The Eye. Have downloaded a lot of stuff I haven't found elsewhere. I was also amazed when I found out about it that it was operating openly. But you were just shut down by a group of religious nuts that didn't even bother using DMCA. What happens when N---o or A---i come calling over complete ROM sets? Those have just become big business again, and can't be considered abandonware anymore. Or M---t over software packages costing hundreds or thousands of dollars each? I'm willing to bet you get a lot more traffic due to the 'questionable' content then the other.
It's your website, you can do what you want with it. I for one never mentioned TOR, and don't think that's a viable solution. But considering the recent downtime due to upgrades, and now getting shutdown by Jesus, alternative distribution options are not the bad idea you seem to think they are.
https://i.redd.it/h2k0pvxpg8x21.png You see its a picture in a library. Not a person with a gun killing anyone.And you know what they say about libraries. Ban anyone that says anything against you or talks about tor that one time but never fucking bring memes into it
This is how you conduct yourself.
I'm sorry but I'm done talking to you at this point, I personally haven't wrong you in any way.
Ever considered drivepool and having multiple duplicates for striped reading? or mergefs for linux i guess.
Just got banned from here for sharing a meme in chat.
93.3k
Members
251
Online
**Welcome to /r/OpenDirectories** Unprotected directories of pics, vids, music, software and otherwise interesting files.
Requests are not permitted. Requests for anything other than technical advice are not permitted.
Try r/DHExchange instead
If a directory is found to contain personal information, child pornography (including animated CP aka lolicon), or any other questionable content. The link will be removed at the moderators' discretion.
NOTE All users should be wary of any open directory or torrent of porn, as well as be familiar of the local and national laws regarding porn, firearms, subversive content, etc in their location.
WGET
WGET A Noobs Guide (PDF) by regravity.
How to install WGET on OSX by nucleocide.
Related Subreddits
How to Threads:
- all resources i know related to open directories
- Googling Open Directories.
- How To Find Almost Anything You Want.
- How Do I Find Open Directories?
- How Are You Finding These Directories?
- How Do You Search For Open Directories?
Open Directory Search Pages