Infinite Fun

So the current migration has been scrapped. I'm turning off the event horizon banners. Lets talk about what happened tho before someone kills me.

I was informed that 8chan has 8 servers with 8 cores and 8 gigs of ram each. However, I was not told that theses "cores" were logical cores (Think HyperThreading) not physical cores. This means it is far less powerful as I thought it was.

This oversight is my fault. I should have more thoroughly checked my environment before making assumptions with configuration based on it.

So, when OdiliTime offered to host a test copy of Next on his own machine, I was blown away that what appeared to be an identical box handling /v/ and /b/'s load and dishing out out 60 requests per second without breaking a sweat. When he looked into 8chan's problems himself, he quickly identified the miscommunication.

We have this limitation because we are using 2ch's hardware. 2ch is a very simple system with static content. It uses a lot of caching and handles a lot of requests (like, thousands a second) without really using any processing power. These borrowed 2ch machines are designed from the metal up with the idea of serving from memory as many static documents as possible. Kind of like 8ch's current software, but without ever writing anything to do the disk besides Oekaki images.

Next is. not. like 2ch, and while its dynamic templating is robust (and pretty fast, for what it is), it's much more reliant on CPU than 8ch is.

What this means

In short, Next has to be at least twice as fast as I originally thought. This is a bit more complicated than just saying "20 requests per second". If the requests aren't being handled fast enough, they begin to pile up on each other, which creates a massive queue. A wonderful visual representation of this is here: http://poal.me/sim1.html

I'm going to accomplish this writing my own caching facades for Next. Specifically, I need to be able to serve at least 50% of responses without ever having to talk to PHP. This begins to shape Next into a lot like how 2ch operates so that we can use their hardware in the same way. There are many ways of accomplishing this, but Varnish seems the most viable. Varnish is a layer between the webserver and the fastcgi; it's whole purpose is to cache responses. It features a templating system called ESI that will designate certain blocks as dynamic, and will make requests for those blocks.

In a typical response, very little of the document is dynamic. In this thread, three points are.

  1. The global navigation will soon feature reported posts and unread PM notifications.
  2. The post form has capcodes.
  3. The posts themselves have an actions list that can include mod actions.

Requesting clients can effectively be classed into four groups.

  1. Anonymous users, the vast majority.
  2. Tor users (which sometimes have different restrictions).
  3. Logged in moderators with no specific permissions for that page.
  4. Moderators with access to different parts of the page.

I have thus far been reluctant to add this sort of heavy caching because my tests have indicated Next is fast enough to not need it. Adding this kind of caching incorrectly can create page freezes and mysterious 404 errors that Vichan and Lynxchan suffer from. I must be very careful when implementing this to make sure that everyone gets the correct views and that they are updated appropriately.

Answers

Why not ask 2ch for more or better hardware?

8chan does not own 2ch. 2ch owns us. We're lucky to be using their hardware or bandwidth at all. I've been given tools and a task and I cannot ask for more tools. Unless the reason for asking for more hardware as a substantial growth in users, I don't think we could bargain our way out of this.

Money

Because of development problems I have closed fundraising until such a time 8chan is using Next. I don't think I can feel good about accepting other's faith when I have lost so much of it myself.

Show's over! Off to master4end76chan.pl!

Yea yea yea. Shills and parasites have popped up all over the place this week because people are justifiably upset and frustrated with the failed migration and they want to take advantage of that. Users feel alienated from their webmaster, and are confused and upset with this fat belligerent prick who's somehow taken his place as the most visible figure on the site. The lack of communication from the top is scaring people and it reminds them a lot of how 4chan was when Moot decided he hated his userbase and started letting his mods make all the decisions.

8chan a large responsibility now and not really a toy. All the glamour and fun of being Based Hotwheels has faded as the site has matured and the reality of just how broken 8chan is right now set in. There's really very little he can do to fix it, and posting is now such a potentially dangerous task I don't blame anyone for not enjoying it. People continue to post on 8chan because they love it, and they love what it represents and what it could be.

Other technical myths

"Next queries the database too much!"

The database handles the load fine. It was the first thing I ruled out as it was the problem last time.

"PHP is the problem!"

infinitydev.org with 1 core and HHVM could handle the combined traffic of all of beta at once and still send pages in 80~400ms depending on load. I'm very confident in my language selection still, but when you're that gimped on computational power everything starts to break.

"FreeBSD is the problem!"

Odili's server ran FreeBSD and it worked fine

Closing

I would like to apologize specifically to /v/ and /sp/. /v/ is a group of people who just want a stable community, and using them for testing purposes conflicted entirely with why they use 8chan. /sp/, on the other hand, just wanted a site fast enough to work. When the first migration failed I shut down the beta and purged them from the servers, then skipped off to a Japanese vacation without offering them any alternative or place to go. This was horribly inconsiderate and I feel that them leaving as a direct result has taken from the community a close circle of well tempered friends that were worth having around.

One more time.
Josh