Hi guys. This isn't so much an image board I'm doing, but a repository where you categorise and organise stuff.
Backstory: I've often thought at times that it's nice to have an organised image collection, but it's hard work. Do I sort them all by series, and then by other categories? What about this image that's both ecchi, but also has maids? What about a situation where you've got multiple characters where it belongs to no single series.
What I've devised isn't a perfect solution, but I think it could be practical and very nifty, if it's not too cumbersome. In short, you import a bunch of images into the system, and you can set flags for different attributes. There's about 20 such flags in the system as it stands, but that's extensible as far as your imagination wants to go. As you'll be able to see, I think I've got a fair number of "useful" ones covered, such as maid, miko, megane-ko, pantsu, gothic, etc. There's scope for a lot more, too.
Of course, all this tagging is of no use unless you can find what you want. So naturally you can just check a couple of boxes and say what you want. Example, I can immediately tell you that I have no miko+megane-ko images, but there are two maid+megane-ko images (and incidentally, they'll both show up if you check the box for "bondage" as well ^_^).
Another nice thing is that you can select great swathes of images and set flags across them. I mean, when you've just imported the latest volume of Muragami Suigun's F-ism collection, you don't want to sit there and tag each image manually, do you?
Anyway, I'd like to offer this to people to see what they think. There's no great restrictions on what you can do at the moment, so you can see all the features at work. You can add and delete categories, you can drop images, you can set flags en masse. And by all means do so. Naturally, I've taken a simple backup so I can fix things if people want to be silly buggers.
http://furinkan.ath.cx:8000/imgdb/
There's the database, please feel free to mess about with it. In the interests of saving peoples' sanity, I've limited the main page to display no more than 50 images at a time. My outgoing bandwidth sucks, and I'm hosting this at home. The thumbnails are small (~4K each) , but things will take long enough as it is.
What I'm looking for:
Any suggestions for improvement will be welcome. I know I should add lots of categories, and I'll do that as I think of them. One thing I'd like to do (and will need to figure out) is to have a hierarchy. Example: when you check "pantsu", the options for striped, spotted, etc should be available. Implementing this will be a matter of consistency, as having a sub-flag set must ensure that the higher up is set, and so on. I'm still learning, so you'll have to bear with me. Also being able to tag an image to series/serieses would be very nice, if difficult. I'll see...
Also, if anyone has an idea for a prettier interface, you could knock up a real simple screenshot of what you have in mind and perhaps link to it from here. I'm a competant web-coder, but I have practically zero aesthetic sense.
Please excuse the bits of SQL lying around everywhere. They're to help me figure out what I'm doing, and whether my selects are working.
For the technically inclined, the database uses some fairly simple PHP to do the frontend, and the backend is PostgreSQL. I was using MySQL before, but I felt I should be using a "real" DBMS for this. It's something new for me to learn as well. Also, I understand that Postgres will allow me to do some stuff that isn't supported in MySQL, such as subselects, and that would be invaluable/necessary when I implement queries like "megane-ko AND maid WITHOUT bondage". It's all running on a box called shirayuki, which you can see if you go up a level from the DB.
Have fun!
It's difficult to hold an open beta when it's that slow.
Ehe, sorry, there's not a lot I can do about it. The hosting that I do have access to is limited (ie. I can't make another table to run this thing, and that's a MySQL server anyway), and is something someone has offered me out of gratitude, so this is pretty much all I have.
FYI, I live in Australia. Our ex-Minister for Communications was the guy who proclaimed that broadband takeup was fuelled by demand for pornography, and essentially relegated us into being a technological backwater as far as internet connectivity. Stupid retard.
Anyway, I'll change shirayuki's bandwidth limiter to allow you a bit more outgoing bandwidth. Hopefully it'll help a little. My upload here is a pathetic 128kbps, and I need to courteously share this line with my family. I can't use all the upload, otherwise it'll slow them to a crawl. I'll adjust the limit on the number of results returned even lower if anyone thinks it'll help.
> FYI, I live in Australia. Our ex-Minister for Communications was the guy who proclaimed that broadband takeup was fuelled by demand for pornography,
Way to prove him wrong! Kidding, kidding.
Anyway, some thoughts, in no particular order:
Further expanding on the virtual filesystem idea, if you then decide to add uploading of images, you can let a user upload a file into /megane/nekomimi/, and the script will automatically set these tags on the new image.
Minister for Comms: Etto... well, he's right in some sense...
SQLite: Hmm, I might take a look at that. One of my priorities though is that it should be scalable and fast. I understand there's a certain elegance in (what I assume is) flat-files for database, but (correct me if I'm wrong) I understand they won't scale well. I'm willing to incur some performance overheads on the small scale if it'll stay relatively consistent up to a larger scale.
Yeah, I dunno what exactly to do about the mid-size images at the moment. They are an ugly hack, and it's especially nasty on wallpaper++ sized images.
Input: Yep. I'm generally pretty concious of security, and I know cross-site and SQL scripting is some of the worst stuff you can have happen. I don't think anyone's entirely immune, but we can do a lot to prevent getting shot in the foot. I'm pretty new to PHP and SQL, so this'll be something I have to learn about.
Path info: I must say, I like the sound of this a lot. Useless? Perhaps, but it could make peoples' life easier. The SQL queries will no doubt be equally ugly, but it makes for a nice and accessable interface. Could you tell me a little more about this thing? I haven't yet ascertained if it's a PHP or SQL thing, or how it'd work...
Thanks for the comments. I'll keep trying to tune and update things.
>>5
Funky! Yeah, I'll add upload support sometime, when I figure out how (and how to do so safely). I don't need people trying to exploit gd.
＿＿＿_∧∧ ／￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣
～' ＿＿__（ ﾟДﾟ）＜ I DEMAND MORE PORNOGRAPHY!
ＵU Ｕ U ＼＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿
You want more? Yeesh, okay... lemme just crack out F-ism 1-9 and dump them in. :)
Or if you want, I'll give you an FTP account to dump in. :P
Furi notices the new and tasty category added (presumably) by Waha
Arg, the blinking! My eyes!!
At least make it CSS compliant... ^_^
SQLite is pretty fast. You're not going to hit any performance problems with it unless you run a REALLY huge site. It's faster than mysql at a lot of things, too.
Also, a good first step for security is to run htmlspecialchars() on all input strings, always.
PATH_INFO is an environement variable that gets set by the web server whenever someone accesses your script like .../script.php/stuff/etc/. I forget exactly how to access environment variables in PHP, but I think it's something along the lines of $_SERVER["PATH_INFO"], which would in that example evaluate to /stuff/etc. It's then up to you to figure out what to do with it. In this case, parse the keywords, and construct a query from those.
Use htmlspecialchars() with the ENT_QUOTES parameter to remove single quotes that can mess up SQL queries and double-quotes that can mess up HTML generation. Right now, doing anything with the "Broken queries?" category fails because of a ' in the tagName.
>>3
Yeah. It's a cool idea. I just can't stand using the thing right now.
Where's the next page button?
You should make sure magic quotes are disabled to avoid really weird bugs showing up.
>>11 Interesting. I guess I should give it more serious consideration.
htmlspecialchars: Ah, so that's the one I want. Yeah, another consideration is that if/when I allow uploads, it has to ensure that those filenames are clean. I don't want to "accidentally" run sha1sum bleh.jpg;cat /etc/passwd or something equally insidious. I think I can cover that, though.
Path Info: Ah, cool. Thanks for the tip. That'll be a bit tougher to perfect, but it's nice.
Next page button? Etto, I'll hack one up for you... it'll be broken, but will have the basic functionality.
Magic quotes: I'll learn about that.
Thanks for your help, everyone. I'll try and fix these nasty bits sometime today.
Magic Quotes is one of the most wonderfully shitty ideas I've ever seen. Not only does it apply strange transformation to all sorts of text that doesn't need it, but even if you wanted to use it, you can't rely on it being turned on anyway.
In that specific case, you shouldn't be calling out to system() ever. Read the file yourself and use sha1().
>>17 Gah, just my luck, it's there under my nose. Hmm, now to see if there's a safe/r 'ls' in there...
Thanks!
This is a lot sexier then the filename hack/shell script I'm using to sort my images. I really want to set up one when the code gets released, except Dreamhost runs MySQL not Postgre. :( SQLite I can't find anything in the knowledge base about, but they have PHP, so it ought to work... Please switch! (or offer a choice between the two)
In the way of actual constructive commentary, the ability to let users rename files might be nice.
>>19 Man, this is all just a matter of knowing where to look... thanks, mate.
>>20 Hehe, in it's current incarnantion it could run on MySQL with minimal changes. I was doing that a little while ago before moving to something better, but I may be able to reconsider that if stuff like nested queries and stuff work in MySQL (I just have to figure out how).
As for the renaming, one of the issues I have at the moment is deciding how to treat the images. It should be possible to handle it if you throw a folder in the source directory, but I'm not sure how to do it yet. Every file will (should) hash uniquely, but if I move every file to a single directory, filename clashes will be possible. I can rename each file uniquely, but this would perhaps mess things up for users. If I find a way to handle folders, there's then an issue with duplicates. If you have a duplicate, it'll be ignored during import. Still, this'd only be a problem if you're keeping manga in there or something.
In short, the system is quite nice for images you've saved off 4chan/wakachan. ow to deal with limitations and further possibilities.
here's an idea. rather than making fields for a limited number of categories that you'll undoubtedly want to change/add to later, make each pic have a variable-length KEYWORD list (take a look at the 'SET' column data type in mySQL).
although, like the fixed number of categories, this has some inherent problems, such as people submitting things with mispelled or variant keywords. (there are a few ways to help fix that, like display a list of similarly spelled keywords, or all keywords, or make some filters.)
Just to let everyone know that progress is being made, but it's more on the backend of things. Things that are much better, like not using exec() when there are PHP functions already there for me. :)
Now with less faggotry.
Let's get this thread back on course, then.
lol at someone administratively losing it. :)
∧＿∧ ／￣￣￣
（ ´∀｀）＜ Look who's talking!
（ ） ＼＿＿＿
｜ ｜ |
（_＿）＿）
So sorry to have ruined your little wankfest, but this thread actually has a purpose, which was discussing image organization scripts. You are welcome to join in! Let me give you an example of how this works:
"Hi! I've made a similar script! You can see it at http://domain.com/path/. What do you think?"
>>31
you are aware that i did exactly that?
i posted a link to tma.pl, explained some of what it does?
well, of course you deleted the posts where i did that in your little private war against someone who is not me.
so, just to repeat myself ...
TMA does technical image matching by comparing the sha1 of the first 4kb of each image (to do truncation resolving) and by distance comparisons between 48dimensional idvector-locations assigned to each image by scaling it to 4x4 pixel and exporting it as RGB.
(perl, cgi, mysql, imagemagick, fuse)
http://baka.no-ip.biz/src/lootie/pubweb/tma.pl
if this makes any sense to you, feel free to join #tma on irc.freenode.net
Are you there, >>33? I have to let you know, there is a fool who has fallen in love with you.
It is me.
I am not a fag as you might think. I have no interest in homo things.
However, something began to swell greatly in me after I saw your posts.
Without even thinking, I came across one of your posts while I fapped.
Ah, even to me it was unpleasant. However, on a certain day I read one of your posts and started fapping.
From the beginning to the end I thought of you.
74 times... this is the number of times I have come reading your posts.
Perhaps, you think that you will reject me. But remember this: I will always be. near you
When you feel someone's hot breath on your neck on a crowded train, it is me.
When you notice someone staring at you in a strange way on the other side of the aracade, it is me.
When you pick up the phone and can only hear panting, it is me.
While you read this, I had already started...
>>33
Where can I see a demonstration of this script with relevant details for some existing pictures?
Hmm, this is most interesting... I've thought about fuzzy image-matching things before, purely for getting rid of "almost-the-same" duplicates. This is a rather cool idea, and certainly far better than doing some sort of edge-detection then matching up lines.
I can see this'd be useful for archiving a place like 4chan. Far too much duplicate crap to worry about...
Why are you using RGB searches to match images that are probably JPEG? Use YUV 4:2:0.
There's a lot that could improve on that. Using YUV, using the distance from the average instead of absolute values, normalizing...
>>41
actualy, "normalizing" is the problem that makes the first about 200k images in the DB less than good. at least what ImageMagicks idea of "normalize()" is. i ended up with idvectors distorted in different ways, depending on the input file format. (no, i dont get that either. but then again, IM is about as readable as my code)
using relative instead of absolute values for comparison is something i have been thinking about, same goes for greyscale of higher resolution than 4x4 or 8bit depth (would be very useful for matching manga/doujin pages).
i dont really understand what the advantage of YUV would be though, mostly because i dont really know what the difference would be.
currently i am holding back on dumping about 60-65 GB more images that are in a holding partition into the DB while i am working on the image-processor, stuff like alternate id-vectors is on the list of things i would like to figure out to avoid reprocessing even more than i have to by now already.
another is somehow converting embedded data like EXIF records into something usable...
ah, so much to do, so little time.
With "normalize" I wasn't talking about anything you'd do with ImageMagick.
Either way, I'm not really interested in helping out with this, seeing as how you're such an annoying and arrogant asshole.
>>43
Normalize
transform image to span the full range of color valuesEither way, I'm not really interested in your "help" there, seeing as how you're such an annoying "i dont know what it means but let me spew some random jargon i picked up somewhere anyways" asshole.
You want to turn this into an academic pissing contest, or what? Yes, I know that ImageMagick has a "Normalize" function. What I was talking about was normalizing in a much more mathematical sense. In your case, for instance, normalizing the distances from the average for each channel against the square root of the sum of the squares of the distances. There are probably other normalizations that might be useful too, that's just the first one to come to mind.
Also, you want to use YUV because it gives much more useful information in each channel, and you can weight the channels depending on whether you are more interested in brightness or colour differences.
> i dont really understand what the advantage of YUV would be though, mostly because i dont really know what the difference would be.
What >>45 said. Also, JPEG is already YUV so you don't have to waste time doing a colorspace conversion.
>>46
actualy, doing some more reading on the topic, it seems that JPEG uses a YUV-like system, but with a full 8bit-depth instead of the clamped value-range of real YUV.
so i end up converting all JPEGs to YUV anyways, or converting anything else to JFIF-YUV.
i mostly stay away from the video colorspaces since i have much less actual experience with those than i have with RGB.
also, a small "waste of time" that happens once per image while i have them open/loaded for rescaling anyways seems to be acceptable there. even more so when i see how making imagemagick not give me a "free" RGB conversion while opening anything might require more of my time than converting a lot of images does require _cpu_time.
otoh, the luminance part of YUV (or jfif-yuv) seems to be a step towards the "greyscale matching for manga pages" thing. but i wouldnt need any color values for that, just something like a 7x7 grid of 8bit depth (so i end up with a comparable amount of work/data as with the 4x4 RGB).
so considering that i can also do channel-based weighting on RGB just fine, i still dont see much point in using a YUVish colorspace for the comparisons.
To threadstarter: Any progress on your program? Code release, compatibility with MySQL/SQLite?
I told a bunch of people I'd be putting my image collection on the web and then discovered ochiba doesn't do what I want, so 困っている. orz
>>49
Eh, well, in a rather embarrassing turn of events, I was going to work on it a few nights ago, and managed to perform an 'rm -fr *' on the damned directory it was in. :(
I'll just have to code it from scratch again, which won't actually be too bad. It was relatively simple, and it'll be faster to code the second time 'round.
Ah, I was wondering why the site went down all of a sudden.
This is one of my reasons for releasing things as open source, as a matter of fact.
>>52
:P I hear you, alright. Tell you what, I'll try and package the php files that I've worked on every now and then, so if anyone's interested, they can attempt to decipher my awful code (I'm a php n00b, but that's why I'm learning).
I've been working on it, and the code's coming back slowly. If anyone wants to check it again, it's still there at furinkan.ath.cx:8000/imgdb/
I've tightened the checking on stuff like user-data insertion (eg. category names), so if anyone happens to break something, I'd love to know. I'll try and get some src up when I next work on it.
Now, something I'd like to ask people who might actually want to use it. I've used a bit of PEAR so far, so as to make the code a little "nicer". I'm also considering making more of the code OO, for the hell of it, but I'm concernded it'll break under PHP 4.x, which is probably what most webhosts would have installed (I don't know if you people just run servers at home, but if you pay a 3rd party host, I assume they have PHP4, MySQL and goddess-knows-what installed.
So. Can I use PEAR::DB, or do you want "hardcoded" SQL statements? I've also made use of PEAR::HTTP_Upload. Naughty? If so, I assume I can just copy and paste the code in from the package, it's just not as "nice" as having it installed with PEAR.
In short, if you can give me an idea of what sort of setup it needs to run on, I'll try and work around that. On my machine it runs with PHP 4.3.10 and Postgres 7.4.7. I'd like to use inheritence in Postgres, but I bet MySQL will break badly even thinking about stuff. I know people want some SQLite love, so I'll try and look at that too, but it'll be much less robust than a "proper" RDBMS; be warned.
Anyhow, any suggestions/criticisms are welcome. Apologies for the fact that as I write this, there's no "tagging" support (I spent a lot of time researching issues and making sure users can't do stupid/malicious things, as best I can). Oh, and there's no thumbnails yet either (that's my next step, once I can make it robust), so be prepared to wait a while if you want to see images (it's been resized on-the-fly in the HTML to 100x100).
Speaking of that, one last thing? GD or ImageMagick? What's supported by most people, and are there any compelling reasons for one over the other?
Thanks, guys.
I've noticed GD tends to make some ugly-ass thumbnails, at least if you're not careful. Maybe one can work around that, though.
Hmm, I've heard such things too. Well, they're thumbnails. It doesn't really matter if they're arse-ugly. That said, if you open more full-size images because you can't judge the thumbnail, that's a nono.
I've heard it may have something to do with the gd library version being >=2. Will have to investigate.
I went to that site, but there was only this place holder page. Nothing else would show up. Are you having domain problems?
>>57
Umm, if you read the post, you''ll see he was berating another user for being an idiot, and then provided an example of how one should post...
Perhaps this is more appropriate, for various reasons:
http://www.example.com/path/
Why doesn't that one work? See here: http://www.example.com/
It's odd using AOL - http://www.example.com is DISALLOWED FOR AOL MEMBERS....
After spending some time hammering away at the code, I think I've finally got this thing back to how it should be. I'm also an impatient bastard, so I paid to get a domain and hosted service. Now I can attempt to show this to people without things slowing to a crawl. I think. ^_^;
Things feel a bit slow at my end, but that's likely just a matter of geography (the server is located in Chicago, USA).
Significant changes are that now search criteria can be inserted via the URL. This isn't bad, and it's something I implemented after suggestion from Waha(?). Somewhere earlier in this thread. Anyway, it seems good, but it tends to break stuff. Want to use the add/delete links at the top of the page? You'd better not have anything after index.php on the URL...
Seriously, working around that stuff is hard. I might just go back to GET queries. It's really quite infuriating not knowing where you really are all the time (which is why you need a clean URL before you click one of the add/delete links).
Apologies for the lack of actual images there at the moment. I've pretty much only just got the hosting and stuff setup this evening, then I had to hack the PHP so it'd work at the new location.
Things I'll add next includes the ability to mass-tag (which is arguably the most useful feature of the damned thing) and characters/series. That way you can do searches for all images with Aria from Sister Princess. sigh of pleasure
Have fun (I hope):
http://www.meidokon.net/index.php
------------------------------
"Meidokon?"
Good question. While I can't find the little essay I wrote about this and "soft", the long and the short is that it's like "lolicon". Thus, Maid Complex. Usage is the same as for lolicon, incorrect as it is.
A site full of meidokon --> Correct!
I am a meidokon --> Correct!
He has a meidokon --> Never!
If you're easily amused, you can go and whois the domain for some rego info. ^_^
Another update:
Stuff has been cleaned up somewhat and some of the suck has been removed. The searching should be pretty damned intuitive now.
It's back to GET queries for searching, because they're nowhere near as messy to handle as PATH_INFO strings. My sanity just breathed a sigh of relief.
What can I say..? There's a terse changelog on the site, and I'd almost call the damned thing usable now. New step is characters and series.
Are you going to rename uploaded file names? Or truncate them, perhaps?
>>64
I'm considering it, yeah. I might do that if the length is greater than n characters, maybe n=20.
And when this is done, what then? This doesn't seem like it could be run on people's computers, purely a web-based script.
>>66
Well, that would depend on what kind of person you are. My machines are already set up to serve a web-based script. :)
Really, it's best suited to an open forum situation where people can add images. Sort of. The reason I made it web-based was mainly because I can show it off, and because it was fairly easy for me to do. I'll admit I've never developed real applications for a desktop before.
If you're interested, there's a piece of software I've heard about but not tried for myself, it's called ImgSeek. you throw images at it and I think you can put them in groups, a bit like my system. One of the cool features it has is heuristic query by example. The example they show you is a picture of a sunset. The system then goes and finds all pictures that it thinks look close to the source you supply. I think that's pretty cool, and it's similar to what someone described earlier on this thread. Anyway, it's free software, and it seems to work.
Main reason I stopped by was to drop another quick update. There's a few more entires in the changelog, and it now supports series marking. In a clunky fashion. I'll have to overhaul the interface part sometime.
You can set the series for a single image after setting its attributes. You can set the series for a bunch of images by marking them, clicking the button as usual, but then ignoring the attribute table. It's ugly, but it works. I'll overhaul the interface and split attrib/series tagging apart in the future.
I also got bored, so the default page is now index.html instead of php. You get a disclaimer thing and a cookie I hacked up. Wai~! I've never used cookies before. They're so easy! :)
Oh, and there's source now. Ehe, I meant to say that, but then I noticed anon had written something. Yes, there's a small link on the splash page where you can grab all of the ugly code I've written in a gzipped tarball. Have fun, suggestions are welcome.
There are some categories that mean the same thing but have different content and names.
Examples:
Loli/lolita
Catgirl/nekomimi
Should that be fixed?
about >>69
I was talking about meidokon's categories in case you didn't know.
I could fix the problem.
I think I'll just fix it...
Yeah, it's something that's been considered. Loli and Lolita have been planned for merging, as can be seen on the to-do list. Catgirl and Nekomimi are different things, though, but would be better handled in a hierarchical fashion. I'd accept mathematically that "catgirl->nekomimi". Ie. catgirl practically enforces nekomimi, but NOT the other way around. How you want to handle it, though, is open to argument.
I see you got rid of Nekomimi. Fair enough, I just pray to Goddess that:
1) You gave all the "nekomimi" pictures the "catgirl" tag first
2) Made good use of the mass-tagging feature
As for your comment in >>69, the categories have different content and names, but they certainly don't mean the same thing. Anyone who is confusing Loli and Lolita is a fool. I did put rollover info on them as well with <abbr>, but I can accept that there are browsers that won't render it. That said, I'll merge Loli and Lolita because they're very hard to effectively categorise data with.
Nekomimi and Catgirl aren't the same thing either. Hazuki (Tsukuyomi ~Moon Phase~) wears cat ears. But they're plainly not real. She's not a catgirl, she's a vampire. Hence the different categories.
>I'd accept mathematically that "catgirl->nekomimi".
>Nekomimi and Catgirl aren't the same thing either. Hazuki (Tsukuyomi ~Moon Phase~) wears cat ears. But they're plainly not real. She's not a catgirl, she's a vampire. Hence the different categories.
Sounds like "nekomimi->catgirl" is truer rather than the other way around, as it can include both natural and costume ears.
-> (or =>) is implies, that's different from > or contains. => catgirl->nekomimi is correct, while nekomimi->catgirl is not.
Uh... "implies" IS "contains". We're talking sets here, right?
Actually, no, it's the opposite. (I hated this Uni maths as well).
It's not mathematical sets, this is to do with logic (but we'll deal with sets in a moment). The use of implies in "A implies B" means that If A, then B, but it's also important that Not-B does not imply Not-A.
I like this analogy someone once gave me. A implies B:
If you study for the test, you can pass. True.
If you don't study for the test, you can't pass. True.
If you don't study for the test, you can pass. True (fluke/cheat)
If you study for the test, you can't pass. False.
So in our case,
You're a catgirl, you have nekomimi.
You're not a catgirl, you might not have nekomimi.
You're not a catgirl, you might still have nekomimi.
You're a catgirl, you don't have nekomimi. WRONG!
As for sets, yes, it's logical to think of catgirl as a subset of nekomimi, and if/when I implement such things, it may involve set theory instead of logic (well, it's all logic, really). The problem here is twofold.
The use of implies in "A implies B" means that If A, then B, but it's also important that Not-B DOES imply Not-A.
A → B
T T T
T F F
F T T
F T Fis the same as
~B → ~A
F T F
T F F
F T T
T T TAlso, "logics" is too broad a term and is not neccessarily contrary, equal or contradictory to set theory.
>>77
Ah yes, true/T/1. :P
I can't remember what that's called. The converse or the inverse...
>>79
Er, wha? In the first table I see two rows where A=F and B=T.
The second table also has two rows like that.
This is a useful reference:
http://www.omnifarious.org/~eljay/info_ttable.html
>>80
Agreed. Apologies for my poor knowledge of the nomenclature.
Anyway, the point is here that the setting/usage of certain categories may affect the use of other categories. It needs to be dealt with in some way.
> Er, wha? In the first table I see two rows where A=F and B=T.
It doesn't matter. What matters is the truth value characteristic of the expression which is identical for A → B and ~B → ~A ( = T F T T).
PS: In truth tables, you look at the columns for matters of truth or falsity of the given operator or atomic sentence particle. The rows only matter insofar as you want to test matters of validity if the given expression is an argument (if all premises are true in one case (row) then the conclusion must be true as well, otherwise the argument is invalid).
Say what? Are you trying to tell me that your result columns are listing possible outcomes? That's the only way I can see it at the moment.
A truth table is meant to show you the result of an operator acting on inputs. As your table is, there are two distinct rows with identical inputs and a different output in each. This implies inconsistency and/or an undefined nature. In short, it looks broken to me.
But anyway, we're somewhat off topic. In other news, meidokon is still adding features, and that support for characters is nearing completion. I'll just have to sit down sometime and hammer it out. There's probably a better way to handle it, but the source is really ugly and it goes through a tree of possibilities to formulate the SQL query string (are there any category constraints, is there a series specified? If so, have they specified characters from that series?)
A friend suggested a "fun" idea whereby meidokon boards could talk to one another to pass img metadata. An image added to the DB on one board could pull the tags and other metadata across on request, saving effort, presumably. And it'd just be kinda cool.
> there are two distinct rows with identical inputs and a different output in each.
I think the "output" is the middle column, not the third column.
Are you going to add "artist" categories as well?
> A truth table is meant to show you the result of an operator acting on inputs.
Which is the column underneath the operator - which is the same for the operator defining the meaning of the expressions itself.
Okay, then, my apologies. I've always had truth tables as A,B,Out.
Artist: It's something I've been considering, yes. I'll finish work on the characters first, though. For the moment, an artist can be a category unto themself.
Wait... are you assuming there's only one "series" per image? That's not always the case.
I know, but it'll make life very difficult otherwise. It's something I've considered and may revise in the future.
Why wouldn't you want to just use categories for characters, series, and artists? That seems much more flexible to me, and less work. What you want instead is meta-categories: A way to group categories under different labels. So you can just say that category so-and-so represents an artist, and category so-and-so a character. Then you just group them appropriately in the interface.
Also, you might want to turn those category lists into some sort of popups at some point, because it's annoying to have to scroll past them at every page.
>>92 would also make it easier to use the software for other things than strictly anime art. You can just set up categories and meta-categories in suitable ways for whatever you want to store.
True. You reckon it's good to have everything as a category? I'm just a little worried about consistency, but that's more of a vague worry in my head than anything I could substantiate at the moment. I can already see that artists as a category is suitable. I suppose I can chuck everything in like that. I'm just worried things will get "messy" if it's not enforced the way it is at the moment. Ah, hell with it... means I have to get off my arse and make hierarchy (meta-categories as you called them) work. Still, no better time than the present. I was about to go and make characters "work".
This could be interesting, actually. Forming data chains whereby an entry can have a pointer to another entry. This sounds fun already! Rawr!
>>93 Haihai~, yeah, they're a pain in the arse as well. I've been thinking some fancy javascript sometime to tuck them away. My focus so far has been on compatibility, so we'll see how it goes (just because it's useless doesn't mean it shouldn't work in links, dammit!!).
>>95 Oh, you want XML now? Go on, I'm listening.
You don't need Javascript to make hidden lists - CSS will do just fine. A quick google search should find you several tutorials on how to do it. The trick is using :hover selectors and display:none.
Of course, you can create more fancy stuff with Javascript, but then you're limiting usability for people who have it turned off.
Also, I've been playing around with making a similar interface, but using the OS X Spotlight metadata database as the backend. We'll see how that turns out though.
I was thinking CSS too, but support for that can be shakier than JS. I think that IE7 script helps that, though.
There's been some discussion about Spotlight. Something to do with smart folders as well. But, I'm not a mac person, so I don't know anything about it.
Just been augmenting my code and the DB. The DB now has a table called file_to_target that holds all the metadata links. There's three columns, file,type,data. File is the sha1 as always, type is now a small set of metacategories (foreign-keyed to another small table). Data can now hold whatever you want, which in theory is bad because you can't foreign key it to anything you control (you'd have to foreign key to a union of table data). Just have to use PHP checks to do constraint checking, I guess.
The code I just fixed means it should be updating metadata in the new and old locations. Once I have it all nice, I can switch exclusively to the new unified table and dump the rest.
Right now, however, is time for sleep (0400). If I'm doing anything stupidly wrong, just let me know. :)
Categories for everything is a win. If you take a look at how major Digital Asset Management systems work, you'll note they use categories for everything, as well as ITCP/EXIF/XMP metadata per individual image (or video or audio asset).
Righty-o then. Categories it is. When I get some more time to myself I'll shove it all over to the unified system. At the moment the code is a horrible mess of SELECT queries to both the old and new linkings. One thing that gets me is that I can't foreign key anymore. It really shits me, and I can't see any way around it. Dunno if I can create a view of the seperate "targets" by doing a UNION of selects of all the tables for each metacategory. Seems ugly, though. I might be able to solve it with CHECK constraints, but that won't be possible unless I suddenly discover I can use if clauses or something. (something like CHECK that "data" is a member of table X when type is "series")
Well, maybe you'll want to try out a category system yourself first, see if you like it. There's a free one, named Picasa2, that google is releasing for free. You'll need windows for it.
To be honest, Picasa2 is piss poor as a DAM, but it'll give a good idea how the more advanced ones do it.
Now, an online website DAM would be amazing. If you add categories, then maybe versioning one day in the distant future, you'd have that all locked up. OTOH, I really doubt an online DAM for the public would need versioning.
The great thing about such a system is it makes categorizing things easier. Instead of spending days categorizing images yourself, let the monkeysvisitors do it. Well, we hope they will, but I digress.
Yeah, I've heard about Picasa. I guess I should try it sometime. Won't know otherwise.
How do you define DAMs in terms of functioanlity. Anything key features that make a DAM a DAM? Hmm, versioning... it could be done. nastily.
(just noticed that I typo'ed DAM and got FSM. Heh, finite state machine...)
The only real requirement for a DAM is that there's a robust cataloging/tagging/identifying/whatever mechanism. Asset Management is exactly as it sounds.
The versioning is just gravy, although it's crucial for professionals. In the context of an online DAM, I suppose it could be used to stack edits together. For example, all the edits that pass through IItran and /os/, and maybe even originals images and derived wallpapers.
There's also colour management, but that'd be a useless feature for an web-based DAM.
Anyway, I do recommend giving Picasa a spin. I don't like it (I use iView MediaPro), but it's easy to use and leaves no doubts where I'm coming from. It'll also give you ideas on what you may want to implement in your own system, once you start banging up against Picasa's numerous limitations. If you have a Mac, try iPhoto.
The no.1 thing I like about yours is that the metadata is all in an open format, namely SQL. Whatever you do, don't give that up. Professional systems that also export SQL cost thousands to license.
Robust, eh? Well, that's exactly what I want to make it, so we're on the right track. The things I've been keeping in mind so far:
A degree of portability - It ran on MySQL first, because that's what my textbook is. :) Then it moved to Postgres, partly because I could, but also because I have this idea in my mind that it's a Real Database. Let's face it, when the default table type for your DB is MyISAM, you've got issues. MySQL alledgedly has better performance, but it's a tradeoff for other more useful features. Either way, it's SQL. Which is just sane, right? I'm an activist, but I'm one of the last people you'll see using something proprietary or with strings attached. How could I ask people to use it otherwise? It should also be usable with SQLite, with some minor changes to the PEAR_DB, but I don't know if SQLite offers the necessary robustness.
Robustness - This basically means paranoia everywhere I can put it. I figure that anything could happen, and I know it's a Bad Thing to make assumptions. Using an ACID backend is of course a Good Thing. The system should recover gracefully from errors/dropouts at any point.
Performance - Okay, there's a performance hit from using Postgres, but that's a necessity. Doesn't mean I'll write bad code if I can avoid it.
Usability - One of the reasons I've not made the UI pretty is because that would require artistic talent. It's also because it'll probably use Javascript and other stuff that's inconsistent across browsers. Likewise, I'm not using cookies or logins because they're a pain in the arse. (the no-mature-content cookie is a triviality, and the site works perfectly without it)
Modularity - I'm somewhat unsatisfied with how this part is going, but I'd like to imagine that blocks of code can be drop-in replaced to enhance existing sections. I understand that not everyone has ImageMagick available and whatnot.
The idea of versioning sounds interesting, but I'm not sure how it'd apply to a situation like mine. Perhaps I just don't know how the DB could be used in such circumstances.
Well, while I quite agree with using PostgreSQL (ask WAHa about his hate-hate relationship with MySQL), by robust I meant a category system that's sufficiently flexible and extensible that it can handle a near-arbitrary number of classifications on any single image. Conceptually this is pretty easy:
Main
Ref # Category Image etc...
1 2 1 ...
2 2 2 ...
3 2 3 ...
4 3 1 ...
5 1 4 ...Have Category serve as a foreign key to the Category table, and Image serve as foreign key to the Image table. Ref is the primary key, Image is not, etc. In order to find all images in a category, you'd do something like a select * from main where category = 2, and to find all categories for a particular image you're do a select * from main where image = 1.
Attributes that are specific to an image, or other image-specific metadata can be stored in the Image table. Same with Categories, etc.
Now, optimizing this may be a bit more complicated. The Main table could get quite big if you're dealing with many images and categories.
Versioning is pretty simple, actually, if you decide to go the everything-is-a-category route. Use hidden categories that group versions of the same image together. They're exactly the same as user-created categories, except only the system uses it.
Forgot to mention: while you'd use hidden categories to group versions of the same image together, you'd put the version ID in an attribute in the Image table if you're dealing with explicit versioning. It's not enough to know a bunch of images are versions of each other, but also in which order they were made.
New Robust: Ah, I already do that. :)
The table file_to_cat looks exactly like you describe, except that there's no artificial primary key. The file and cat column together make a primary key, and this has worked nicely. As for optimisation, well, it's been nice so far. I had an absolutely mind-blowing moment some weeks ago when a friend told me how to make a hellishly elegant query finding multiple entries of img X with category A,B,C set. I think it might even scale linearly, but it's not something I've investigated. It's nice, though.
Versioning: Ah, I see. That's simpler than I thought it might be.
Quick update. The backend has been generalised a lot, meaning that with a little more work, it could easily be adapted to most situations. As it is, it's using a fixed set of meta-categories. Handling a variable number of meta-categories will require extra work, and is actually somewhat undesirable in these circumstances. Characters have ties to series, and a more "vanilla" implementation wouldn't have such linking (I think).
Now that an image can belong to multiple series, it'll take a little more work to figure out how to deal with characters. I'd display all possible characters on the index page, but it's huge and ugly. I'll... work some interface thing out.
ok, so SQLite decided to eat my ochiba database. Looks like I'll have to redo all the categorizing; before that, are there any other image repository scripts in the woodwork I could consider switching to? I'd really like one with the ability to search for multiple keywords, and all the 'PHP sucks' talk on 4-ch is getting to me.
http://danbooru.donmai.us/ would work beautifully, if only dreamhost offered PostgreSQL.
>>111
there are two tagging/categorization systems that can be found by visiting irc://irc.freenode.net/tma ... but i guess that was already mentioned here.
and no, i do not mean TMA itself, that one is still just working with auto-generated low-quality metadata. but the authors of other image-db things hang out there too ...
How badly do you need Postgres? Depending on functionality, danbooru might be able to be converted to MySQL. Meidokon could use MySQL with some work, I know that much. You could switch to another host that supports Postgres, but it'd probably be more expensive. We like Postgres just because it's more robust and "proper" about what it does.
Hello, I am a member of Team Danbooru.
If I recall, there are only two places where danbooru relies on pgsql features: the rules on posts_tags (to automatically increment tags.post_count) and the pgplsql stored procedures to perform related post count. The former can easily be moved into the app and the latter you don't really need, you can just rely on tags.post_count instead.
There are plans to offer a SQLite version in the future, but right now we're busy whoring Postgres for all she's worth.
but right now we're busy whoring Postgres for all she's worth.
Amen, brother.