Webhosts/Other software for Imageboards? (88)

1 Name: TheStupidOne!36Pr0Wdt4A : 2007-04-03 15:13 ID:QgF9JsnR [Del]

Hello,
I'm the admin/owner of fatechan.net, and I run /moe/ and /cw/ on iichan. When I first started my boards, I had my own box I had coloed, and everything ran fine. Due to some monetary issues tho, I was forced to cancel my colo and move to shared hosting.

Problem is, my account has now been suspended several times for excessive cpu usage. I'm hosted with cirtexhosting.com, and I run kareha. I keep my thread count around 6 pages (I manually clean out the boards), but according to my host, I'm still eating 14% of cpu time.

Any ideas for hosts that would be more lenient with their cpu usages? Dreamhost is not an option, I'm just waiting for them to collapse from overselling. I use kareha for the custom tripcode catches, so if there's another imageboard soft out there that can do that, I could switch.

Any ideas at all? I'd rather not have to pay more for a semi-dedicated just for the imageboard.

39 Name: !WAHa.06x36 : 2007-04-25 10:41 ID:Heaven [Del]

>>37

Actually, I should specify that I meant that in mode_image, it uses next to no CPU. mode_message invokes the script every time you view a thread, which drives up CPU usage considerably.

One could easily enough change mode_message to link directly to the HTML instead of linking to the script, too.

40 Name: TheStupidOne!36Pr0Wdt4A : 2007-04-26 02:06 ID:MCXL6Ewr [Del]

Then find me a way to safely migrate over 30 pages worth of images in two imageboards from kareha to wakaba. I'm in mode_image too.

Hacking the custom tripcode catches wouldn't be difficult for wakaba, but I already have a huge amount of threads in kareha, and I'd like to stay away from a complete wipe. The loadbalancer coding is kinda fun too anyways, trying to remain as completely drop-in and simple to setup as possible.

41 Post deleted by moderator.

42 Name: Anonymous : 2007-04-27 13:38 ID:Heaven [Del]

It's ok. I just watched the first 3 eps of nanoha strikers, and it seems they they don't fit to your board anymore. 20+ something... :(

43 Name: TheStupidOne!36Pr0Wdt4A : 2007-04-27 14:41 ID:Wlgmlcjj [Del]

Pfft, they just move to /cw/ now =P

44 Post deleted by user.

45 Post deleted by moderator.

46 Post deleted by moderator.

47 Name: Anonymous : 2007-05-04 16:30 ID:5NoZ2zA2 [Del]

>>40
Man, I hope migrating /moe/ and /cw/ to wakaba is successful. I'd hate to see all those threads I haven't yet had a chance to peruse disappear!

48 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-05 01:40 ID:GPJYd31Y [Del]

>>47 I'm not migrating to wakaba. I'm currently coding a new imageboard script. After some analysis of kareha, there was just too much that needed to be modified in order to make it run more efficiently.

Friend of mine wrote a script to export all the threads from kareha and strip the html from them, giving me a base to build a new script off of. I'm going to keep the style of kareha, drop in, very little configuration, and nothing other than a webserver needed to run (IE flatfiles, no sql db required).

Gives me some practice and I find it a fun project. I'll implement all the features I want, while keeping it as simple to run and maintain as possible.

49 Name: Delicious Flat Chest : 2007-05-07 03:40 ID:k9hz7B52 [Del]

So the down time wasn't just me... that's good to know... Hope Fatechan /moe/ and /cw/ can get up and running soon...

50 Name: Anonymous : 2007-05-07 05:20 ID:Heaven [Del]

Since Fatechan's downtime seems to cause quite a bit of confusion, maybe the 403 message should be replaced with a short explanation?

51 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-07 11:29 ID:cjqdclvC [Del]

>>50 Thanks for that suggestion, I'll put up a placeholder thing

52 Name: Anonymous : 2007-05-07 12:09 ID:5NoZ2zA2 [Del]

>>48
So now we'll get to enjoy yet another new imageboard script, instead of the tried and true wakaba. In the meantime, no /cw/ or /moe/.

Oh well. They're your boards, do as you like... I'd like you to go in a different direction though.

53 Name: Anonymous : 2007-05-07 20:17 ID:HYJA+FmD [Del]

>>48
Why not use wakaba or the like?

I'm just curious because I just did the same thing recently (wrote my own script, that is) and I'd be interested to know if your reasoning is anywhere near mine.

54 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-08 00:25 ID:/TncEzmT [Del]

I hate mysql, that simple. And I think it's a real waste of resources to run a full db for something of this nature. This isn't like normal bulliten board software. Content on the boards isn't dynamic at all, once it's posted it doesn't change.

The only time the pages need to update, is when a new post is made, or one is removed. And even when that happens, there's at the most, a change to a thread, and to some of the main page indexes.

There's only 6 different kinds of posts.

1) Post is to an existing thread on the first page, and isn't a sage post, only the thread and the first page need to be rebuilt
2) Post is to an existing thread not on the first page and isn't a sage post, only the thread and the pages from the front back to where that thread was originally located need rebuilt
3) Post is to an existing thread and is a sage post, only the thread and the page the thread is on needs updating
6) Making a new thread, new thread needs to be generated, and all page indexes rebuilt
5) Deleting a post, only the thread and the page the thread is on need adjusted
6) Deleting an entire thread, all page indexes from where the thread was on back need to be rebuilt

The majority of posts occur in the first 3, new threads are uncommon, case 5 is usually an admin action, and case 6 is 99% of the time only done by admins.

But even then, new content isn't that common. On traditional boards the most common state is static, and static content is very easy to push around. Through my friend's (hihi pc486 :V) profiling of kareha, the majority of cpu time is being used by i/o operations. And the main source of that i/o is writing the caches.

After going through the code, kareha actually wastes a lot of time rebuilding threads when it doesn't have to. Also not storing individual posts and the shortened thread displays, wastes processing time as it has to completely strip the html from the thread, and then rebuild them as well as make the shortened version again. I think with adding a few additional cached items, the shortened threads for main page display, and the individual posts, and to follow those 6 cases and only do what's necessary, I think a flatfile system can compete with a db-based board in most cases.

Only advantages a db would give is ease of programming, less disk space taken up since there's no need to cache as much data, and the flatfile system will become very i/o bound on heavy traffic boards. I'm aiming for ease of installation and matinence, and to enable the script to be run nearly anywhere perl and some diskspace is available. Along with being built from the ground up with image loadbalancing capabilities, it'll make it very easy to scale the boards as traffic grows.

If I were programming just for myself, I'd probably just go python or ruby and postgres and not worry about ease of installation or operation. But I'm aiming to expand on the initial concept and utility provided by kareha, and make a capable and more powerful successor, while keeping the ease of use and matinence with a flatfile system.

55 Name: Anonymous : 2007-05-08 04:33 ID:Heaven [Del]

> python or ruby and postgres and not worry about ease of installation or operation

Sounds... exactly like what I did.

56 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-08 13:31 ID:/TncEzmT [Del]

Yeah, if I were coding solely for me, I'd just use postgres and whatever language I felt like. But I'd like to give back to the community and build a viable and worthy replacement for Kareha, rather than like what's been happening here, tell people to install a db and run Wakaba when they run into Kareha's shortfallings.

I think the design of flatfiles have a LOT of potential, and for most imageboards, would be more than enough to run fine. And I like the drop-in ease of the flatfile design, all people have to do is just unpack, change a few settings, and be done. Redesigning the caching system, along with adding native loadbalancing for the images, will go a long way to allow it to run fine on most size boards.

My goal for my script will be ease of installation and use, allowing anyone who knows how to use ftp and edit a few text files to be able to setup and operate the entire script, including the loadbalancing. The loadbalancer servers will just need a webserver and possibly either an ftp account, or even just a dropin script to upload the images to.

57 Name: Anonymous : 2007-05-08 19:51 ID:Heaven [Del]

It's funny, I was talking to a friend of mine and he said just the opposite, and suggested the design of Myspace as a counterexample. Everything on there is database-driven -- all the text, images, etc. The rationale is that it takes more time to reparse a flat file and insert the dynamic content than to keep the parts in separate fields in a table and pull them out at page-fetch time, and the load balancing could be done transparently because the database server connections are all happening behind the scenes.

It makes sense for a half a second until you consider that the overhead of running a database server in the first place is probably greater than any of the IO/processing time required to cut a file into several pieces, and sites that actually see enough visitors to make it worthwhile to run two servers per page view are atypical. He pointed out that, since the DB is "in memory" it's quicker to pull a record out of it... but then again, the file inode cache would be in memory as well. So yeah, I'm beginning to doubt the usefulness of running a database at all, besides for statistical purposes and (as you mentioned earlier) to simplify the code -- but if you're trying to write portable code that can run on more than just MySQL, your code is probably not even going to be any simpler, either.

58 Name: Anonymous : 2007-05-08 19:54 ID:Heaven [Del]

... that sort of rambled on for a while without much of a point, didn't it.

By the way, consider what Trevorchan did. I don't believe "ease of installation" should be one of the key features of an imageboard script, if only to prevent the creation of six thousand useless imageboards built by 16-year-old script kiddies.

59 Name: Anonymous : 2007-05-08 22:23 ID:Heaven [Del]

>>58
I think she's already discouraged that by not using PHP.

60 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-08 22:59 ID:Heaven [Del]

>>57 My thoughts exactly. Very few imageboards see the level of traffic, where the overhead of running a database will be less than being i/o bound by a flatfile design. With proper caching, a flatfile can easily keep up, and in some cases, be faster than a db-based setup. The only major drawbacks are more complex programming, and more disk space used to store cached processing steps.

61 Name: Anonymous : 2007-05-09 18:21 ID:5NoZ2zA2 [Del]

>>57
Plus the web-side component of an application like that can typically be scaled with far less pain than the big ol' xbox-hueg centralized database. It's just more hardware and that's that, whereas scaling a database means more I/O bandwidth and more CPU sockets.

So it makes sense to stick as much of the required processing in the website layer, as long as this doesn't mean doing SQL joins in the application or other database abuse. Caching, too, is usually best done at the web server assuming that coherency is being taken care of.

All of this is obviously immaterial to stuff that just runs on a web hosting account. Hell, I remember the etherchan wiki (RIP) dying because mediawiki, or that particular mediawiki installation anyway, banged the shared database server too hard. ... though I've got a suspicion that the database administrators had calmly dropped some of the indexes that mediawiki required for proper performance, thus provoking unholy quantities of sequential scans.

Also, holy shit are you people polite. Four entirely bumpworthy replies and all of them are saging...

62 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-09 21:16 ID:Heaven [Del]

Yeah, with the futaba-style image/post board, we're not worried about searching, so there is absolutely nothing a db can do that a flatfile just can't realistically.

Once I get a little base of code setup, and a good plan, I'm going to put up an svn server for anyone who wants to donate time and coding effort to it. I really want this to be a viable replacement for kareha, and also address any shortcomings of wakaba, so input from other admins and programmers is very welcome.

63 Name: !WAHa.06x36 : 2007-05-09 22:52 ID:Heaven [Del]

>>62

Just don't add stickies.

64 Name: Anonymous : 2007-05-10 06:53 ID:Heaven [Del]

>>61 it's not about being polite or rude, it's about avoiding the ID. For some of us anyhow. Sage is Heaven.

65 Name: Anonymous : 2007-05-10 09:10 ID:Heaven [Del]

Oddly enough, I've come across a couple of shared hosts where flat-file message boards are explicitly disallowed. Their excuse was they took up too much CPU. It's one of the most absurd rules I've ever encountered.

66 Name: Anonymous : 2007-05-10 15:34 ID:5NoZ2zA2 [Del]

>>62
You do realize that SQL databases don't provide for searching, right? MySQL's non-ACID MyISAM tabletype supports a half-arsed form of fulltext indexing, but that is next to useless without control over things such as stopwords and the like.

As to things that SQL databases can do that flat files cannot, well... if you don't care about (or don't know) the difference between a sequential scan and an index scan, then perhaps it is better that I don't try to sell these points to you. I expect you'll find out on your own exactly why many developers prefer to simply bang out a short SQL query rather than spend a couple of hours writing the equivalent query out for flat files.

67 Name: Anonymous : 2007-05-10 18:29 ID:Heaven [Del]

There are versatile full-text search plugins for many common SQL databases. Even SQLite has one, and a pretty decent one at that.

I don't see this as necessary, anyway, since the design of Wakaba-ish boards makes them easily indexable by web search engines. One of my boards was imported from an old YaBB setup, and it currently has about 3500 posts. I added a Google site search box to it, and it's proven every bit as useful (if not more useful) than the overly-complex YaBB search interface with none of the code -- well, except the HTML, of course.

68 Name: Anonymous : 2007-05-16 20:14 ID:Heaven [Del]

So when're the /cw/ and /moe/ boards coming back? I'm jonesing real bad here.

69 Name: TheStupidOne!36Pr0Wdt4A : 2007-05-18 18:41 ID:Heaven [Del]

I think I'll just get a temp loadbalancing hack in there for now, then I'll continue work on the real replacement script without having to rush.

70 Name: Delicious Flat Chest : 2007-05-21 17:46 ID:wnSf3RRE [Del]

Yeah /moe/ and /cw/ always seemed to have only work-safe material unlike 4chan's (http://www.4chan.org) /c/ (http://zip.4chan.org/c/imgboard.html)... especially after MOOT made the change to /b/... making /b/tards post there stuff... there...

71 Post deleted by moderator.

72 Name: Kankuro : 2007-06-29 15:14 ID:Ytu/zxGU [Del]

Aw man i think the page will never come back to us.
Well i will be looking for other moe work-safe site

73 Name: Yo : 2007-08-26 20:37 ID:Qv9/YaIW [Del]

Fatechan! I miss you! I cannot find another site as great as you! I need delicious flat chest!

74 Name: Anonymous : 2007-09-15 18:54 ID:/wBf7SaO [Del]

Look, I have a shit ton of space I'm not using on my dreamhost account right now. Granted, that 500gb is grossly oversold, but if you're interested, the email address is in the link.

75 Name: Anonymous : 2007-09-17 13:57 ID:iKA0W/0u [Del]

i've been using kareha on shared hosting and i probably get as much use of it as you did if not more. i havent had any issues. maybe you need to find some competent hosts.

76 Name: TheStupidOne!36Pr0Wdt4A : 2007-10-22 07:56 ID:o07pRE3c [Del]

I've tentatively restarted /moe/ and /cw/ on a vps of mine while I still finish my kareha hacks for image load balancing. I just got to make sure I keep my bandwidth limits in check >>;;

77 Name: Anonymous : 2009-01-27 20:00 ID:Q8koOefu [Del]

Hmm..

So i just started a brand new imageboard..

What should i do with it?? where should i get traffic?
andy ideas?? i want to make it as unique as possible..

thanks

78 Name: Anonymous : 2009-01-27 22:59 ID:Heaven [Del]

>>77
you've got things a little backwards there.
you should ask those questions before you decide whether to make a new imageboard or not.

79 Name: Anonymous : 2009-02-02 14:49 ID:nFh+DURc [Del]

>>54
flatfile databases are definitely not doing your cpu usage any favors. If you think running mysql or postgresql (postgres > mysql in most cases) is too heavy for a little operation like yours, I'm pretty sure you can set up wakaba (or pyib) to use sqlite, which should be lighter.

Anyway, take a look at vpslink if you haven't already. They offer tons of customizability, and it's the cheapest vps you'll find anywhere.

80 Name: Anonymous : 2009-02-02 15:49 ID:Heaven [Del]

>>79

> 2007-05-08

81 Name: anon : 2009-02-04 05:59 ID:kstmBZ6v [Del]

fgs

82 Name: Anonymous : 2009-02-28 07:17 ID:nCItJTHZ [Del]

> 2007-05-08

Did anything come out of this? He still using Kareha?

83 Post deleted by moderator.

84 Post deleted by moderator.

85 Post deleted by moderator.

86 Name: Anonymous : 2012-09-01 17:51 ID:8IyM9hwa [Del]

I hate the spam on these boards.

87 Post deleted by moderator.

88 Post deleted by moderator.

Name: Link:
Leave these fields empty (spam trap):
More options...
Verification: