Most efficient file storage layout...

[abudabit]

I want to have image storage on my website and want to plan an efficient method just in case there are a lot of users.

Would it take too long for the system to seach for a file if there are 10000's of files in one directory? Would it be faster if I had a bunch of directories referenced by a random directory name in the database, such as "images/fdasfd123ds/filename.***" instead of "images/filename.***"? Or to take it further, two levels of random directory names such as "images/fdafs/fdafsd/filename.***"? I don't know enough about operating systems to know if any of this matters.

I'm sure someone here has dealt with this before, everyone here seems really knowledgable. Thanks in advance.

[swirlee]

Would it take too long for the system to seach for a file if there are 10000's of files in one directory? Would it be faster if I had a bunch of directories referenced by a random directory name in the database, such as "images/fdasfd123ds/filename.***" instead of "images/filename.***"? Or to take it further, two levels of random directory names such as "images/fdafs/fdafsd/filename.***"? I don't know enough about operating systems to know if any of this matters.

This is actually a pretty good question. But the answer is highly dependent upon which filesystem you're using. Some filesystems are optimized for very large files, some filesystems are optimized for a lot of small files, and some are somewhere in the middle.

I recommend that you first figure out what filesystem you're using. If you're on Windows NT/2000/XP it's probably NTFS. If you're on Win98 or Me, it's probably FAT16 or FAT32, and you seriously shouldn't be using it anyway. Upgrade. If you're on a Mac, it's probably HFS. And if you're on *nix, well, then it could be anything. Ext2fs, JFS, ReiserFS, etc. Once you figure out what you're working with, Google it. You can probably find some good information in the Filesystems HOWTO.

[abudabit]

Unfortunatly I'm using a free test server before I go on to a real host, so I don't have that info yet. I guess I'll do 3 layers of directories with 50 name posibilities each. 125000 different end directories is probably enough unless I get like 50Million files which will probably never happen. And if it does I guess I could change it to 80 name posibilities each. Of course this is all speculation.

I do notice that Windows XP has a problem with hundreds of files in a folder although that might be more of a GUI issue.

Thanks.