Member Controls

Goto page 1 2      Next
Developers > Should I Make An Object Scanner/Organizer/Deleter?
XenosLink to postposted: Wed Mar 29, 2006 2:28 pm

Member since:
 2005-10-04
Posts:
 17
I've been thinking about this for a little while, but haven't really gone beyond the "dreaming about it" phase at all. The reason I'm asking about this before giving it a shot is because I realize that there is already a fairly decent object scanner in SimPE, so I don't want to step on toes or produce anything that will prove redundant.

Basically, I'm thinking about a standalone application that would make use of some of the routines already in the various DLL's that ship with SimPE (to save me development time - upon cursory glance, many of the routines that I need are already in there). The application would be installed into the same path (or a subfolder within the same path) as SimPE so that common code wouldn't need to be duplicated. (I wouldn't be opposed to it being distributed along with SimPE should it be deemed worthy at some point, either).

Here's what I'd like it to do:

1) Scan each individual file in your root downloads folder, identify the file type, and then move it to an appropriate subfolder to help organize everything.

2) Rather than just making a cache, the pertinent data from each file is stored in an Access MDB file. This will greatly speed up processing time and reduce the required memory overhead - especially if you have a lot of downloads - only dependent files are loaded into memory at the same time.

2a) Because installers put things into the root downloads folder, if you haven't manually deleted any files or deleted any objects/outfits within the game, you only need to scan that top level folder - thus making it faster.

2b) If you have manually or in-game deleted something, then rather than scanning and reading each package, you can just instruct the program to run through the DB file and see which entries no longer have their corresponding package file. This should (I think) be much faster than actually reading each package from scratch every time.

2c) Since databases have easy ways to comare data in "dependency" fields, if you deleted something from within my program, it would know immediately which, if any files, were dependant upon that file and could warn you or batch delete them all accodingly.

3) Finding missing meshes could be made considerably easier. There could be an online database somewhere (assuming we could find a place to host it where cost wasn't an issue). Object makers could add their objects to the DB and recolorers could add theirs as well. Plus, I could set it so that when my system finds a mesh/clothing combo, it would provide the important data to the master online database to fill in any gaps where creators haven't added the info themselves.

3a) Eventually, I'd like to tie it into the Google API so that if I have a "missing mesh" I can use the "clues" given to me by the package files and see if Google can actually tell me where the required mesh is located on the web - and, when a user finds it on the web, they can plug in the URL and add that to the master DB to save time for future seekers.

4) Various other integrity checks would be quite easy, too.

5) Resetting clothing/hair and other in-game thumbnails would be possible (I think), too. I can't tell you how many hair and clothing thumbnails were broken when I first viewed them (i.e. I downloaded a sim, but neglected to download the mesh). Then, when I finally did get the mesh, the thumb never reset itself, so I never know which in-game outfit is still missing it's mesh and which has been found without trying each one (and seeing which one it is).

6) Comparing global hacks to the master objects.package etc. we can see exactly which globals/semi's any given package is overwriting. Each global hacks package could be compared against others you have to see if conflicts may exist, too.

----

And, there are obviously other things that it can probably be made to do at some point. SUMMARY: The key points to this are simply - an object/downloads manager that can truly manage files in such a way as to keep everything in neat and tidy order. It also ties into a master DB that makes it easier for others who are missing depencies - if one person does the detective work on an object, then it'll save others the time.

But for this to work, there are certain things I'd need to know:

1) Would people use this?

2) Does anyone know of a place that might be able to host an online database (probably mySQL, but any database that I can remotely connect to would work). Obviously, if I'm not charging for the program, I can't afford any monthly hosting fees. There may be ways to combine access to the main DB with advertising and the like, but I don't have time to manage that - but if you are a potential host and would like AdSense, other advertising, or whatever, I would definitely work out the program on my end so that it could maximize your potential to at least cover your costs.

3) Would there be a dozen or so people who would be interested in being "Power Users" who would help to ensure the integraity of the "human added" sections of the main DB? Obviously, package files might contain clues as to where to find a mesh or other missing element, but an actual link to the download page would have to be added by a human. Things go missing, data entry mistakes can be made, and so on - so there would need to be a certain amount of time spent to make sure that the integrity of the shared DB was accurate. (Regular users could report a "broken link" or whatever, but a moderator type would have to verify it).

4) Would the existing SimPE developers be willing to help me with questions as I get rolling. I'm starting to know my way around package files, but there are still many areas that are pretty mysterious to me. I might see a number and know it's important, but I might not know exactly what that refers to (something in the main objects.package file? another field in another package file? etc.) And so, I'm sure to have several questions as I try to muscle through it.

So, if there are enough yesses to the 4 questions above, I'll likely give this a shot. It sounds pretty complex, but upon cursory glance, many of the core routines needed to scan the files and such are already in the various DLL files that ship with SimPE.

EDIT/ADDED: I also don't know what the plans are for the current object scanner. If this is something that goes against plans for that, then I don't want to screw that up, as well... So, if the author of that has problems with this concept and it's potential to obsolete that scanner, let me know! As I said, I definitely don't want to step on the toes of those who already have plans for things!
quaxiLink to postposted: Wed Mar 29, 2006 3:27 pm
Avator for quaxi

Member since:
 2006-04-28
Posts:
 3154
I'll help with Questions whenever I get the time. So don't worry about that.

And there is no such thing as making work obsolete. It is allways good to have various alternatives. Noone will be angry, if you develop another Scanner.
In fact, you are more than welcome to do so.

I am currently working on a new Version of the "Scan Folder" Plugin. It will certainly not feature all the things you wrote about, but it might provide usefull Routines for your work.
It will certainly be a SimPe Plugin (no extra Aplication), and possible Parts of it gonna be a noew Wizard. So, If you want, you can also help with that development, which is currenty pretty much in the Design stages.

But as I said, If you want to develop something of your own, and need help with package related stuff, I am here to help. :)
nomatxolLink to postposted: Wed Mar 29, 2006 6:36 pm

Member since:
 2005-10-04
Posts:
 3
Hi,

I was thinking of doing the exact same thing as I checked the boards and saw your post. I'd be more the willing to help you rather than start opposing projects.

Let me know if your interested in some help.

Any language is fine for me, esp. C# or C++.
quaxiLink to postposted: Wed Mar 29, 2006 6:54 pm
Avator for quaxi

Member since:
 2006-04-28
Posts:
 3154
SimPe is developed in C#. So that would be the language of Choice.

I am working at the basic Designs for the new Scanner plugin at the Moment. Stuff I want to reuse in iother Parts of SimPe, like an easy way to get hold of the Contents of a package, or generate a 3D-Preview, if SimPE was unable to find the Thumbnail stored by the game.

After that, I am going to rewrite the Scanner Plugin, and redesign the IScanner Interface, to give more flexibility.

So what I need most right now, is suggestions, and Ideas on what is needed in the Scanner Plugin.
If you have Ideas about technical aspects (like, what Methods should a Scanner Interface offer) let me know them as well.

Once we finished the Design, I will implement a reference Scanner. At that Point other Developers are welcome to help implementing new / port existing Scanners.

At the Moment I am not sure about anything, so I can't say what will be needed exactly. But any help is very appreciated!!!
jaxad0127Link to postposted: Wed Mar 29, 2006 7:11 pm
Avator for jaxad0127

Member since:
 2005-10-01
From:
 Near my laptop
Posts:
 1067
Maybe you could just combine your developments and end up with a new plug in for advanced users and a wizard for less knowledgeable ones.
quaxiLink to postposted: Wed Mar 29, 2006 7:46 pm
Avator for quaxi

Member since:
 2006-04-28
Posts:
 3154
That is the Idea. To develop Components, that can be used in a simple Wizard, but also can be reused in the Scanner Plugin for advanced Users.
nomatxolLink to postposted: Wed Mar 29, 2006 8:08 pm

Member since:
 2005-10-04
Posts:
 3
I'll read the SimPE source over the next couple of days, and if I have have a suggestion / needed feature, I'll let you know.

Mmmmm, source, Doh!!
XenosLink to postposted: Wed Mar 29, 2006 10:18 pm

Member since:
 2005-10-04
Posts:
 17
Hi Guys. Good to see some responses in here.

Just so you all know, I develop in Visual Basic - so whatever I build would need to be done in VB. I can read C code, but I could never mount much of an effort to do something from scratch in it.

One of the reasons that I planned for the application to be a standalone rather than a proper plug-in for SimPE is just a matter of overhead. Working with Database files (both local and remote), adding Internet connectivity for mesh searches, and a lot of the other things I would plan to do will have their own overhead. To place that on top of the system requirements for SimPE would either slow it WAY down or require a whole lot of extra planning and considerations that could otherwise be avoided.

As far as teaming up to do stuff, that would be cool. I've never really managed a multi-developer project before so I'm not exactly certain how to go about it, though. There are certain areas where I would surely need help (for example, I would LOVE to have a preview screen where you could load the mesh AND skin it with the various recolors for it - both for objects and clothes/accessories/hair - I wouldn't even know where to start with that, though).

Lemme take a little time tomorrow and start going through the various routines and such within the DLL's that are out there. I think I've found the key ones for file access - I just need to look at various object types and make sure I'm identifying all the key elements that will uniquely identify the object/skin, properly identify the type of package that it is, identify which other dependencies it has, and so on.

It'll probably take me a week or so to get a core application that'll read and identify files. Once I have that, we can figure out how many other folks are interested in lending a hand and figure out who is going to do what.

Obviously, one of the first steps will be to create a directory tree and get the program to accurately move files into the right place (so that they are organized properly and any "must be loaded first" type things are taken into consideration. Another early step will be to have a routine that scans the in-game packages and grabs identifiers from there so that it can scan for bad GUID's, overrides, etc and have the master Database be "self-updating" with each expansion/add-on pack.

Cheers and I'll keep you all posted as to my progress. If anyone has specific ideas or skills they'd like to mention, feel free.
quaxiLink to postposted: Wed Mar 29, 2006 10:39 pm
Avator for quaxi

Member since:
 2006-04-28
Posts:
 3154
As I said standalone is ok with me, this I just Information, I case you did not know. A Tool PLugin (like ScanFolders) does not produce additional overhead for the main Application (ok, it is one more Methodcall, but that is it), as it is only created when the User cstarts it (just like a standalone app).

About classifying packages. Have a look at the simpe.scanner.plugin.dll. It contains several classes, that implement the IIdentifier Interface. You can send each of the a package, and (if they can identify the Type) they will return the Type of the stored data, or unknown if they did not recognize it.
Works quite solid so far, so that might be usefull for you as well.
tickleonthetumLink to postposted: Thu Mar 30, 2006 12:24 am
Avator for tickleonthetum

Member since:
 2005-06-16
From:
 U.K.
Posts:
 108
I also like the idea of this being a plugin for SimPE. Oh and I also like the idea in and of itself we do need some sort of organizer program.
Goto page 1 2      Next


viewthread, 0, 0, Should-I-Make-An-Object-ScannerOrganizerDeleter