Home | Download | Discussion | Help | Site Map | New Posts | Sign in

Latest Site News

MTS Bob Ross Paint-Along Night - posted on 4th Sep 2017 at 10:54 PM
Replies: 5 (Who?), Viewed: 2931 times.
Test Subject
Original Poster
#1 Old 28th Jun 2017 at 3:53 PM Last edited by Londondreary : 28th Jun 2017 at 7:07 PM.
Default Using S3PI (C# library) to modify .package?
Alas, Google was no help with this one.

There are a bunch of clothing mods that have separate packages for each of the ages. One complete package (mesh, texture, skinning, etc) for adult, another for elder, another for teen, etc., and they have the exact same texture files. But if I merge three of these age packages, there will be three sets of identical textures. That's pretty bad, especially since The Sims 3 is a 32bit program with D3D9 (VRAM is mirrored in RAM).

How would I merge these packages, so that they use the one set of textures?

Here's a screenshot to demonstrate the "problem":



Edit: After thinking about it for a wee while, S3PI ought to be the way forward:

Load Package
Find pointer to textures inside the GEOM objects
Sort diffuse, normal, & specular texture pointers into string lists
Compare textures, and remove duplicates
Create two new lists, one for duplicates, another containing the original at the same index
Search GEOM objects for strings pointing to the duplicates, replacing with the original's pointers
Remove duplicate textures

The problem with that, is S3PI has no readily available documentation. The closest thing I could find was this auto-generated nonsense that is of no help whatsoever. The type that goes "class S3PIWrapper is a wrapper that wraps S3PI classes", rather than explaining what something is, how to use it, etc. An example of proper documentation would be FreePascal's docs, Bethesda's Papyrus docs, OpenXML's API docs, etc.

If there aren't any docs, then the API is useless. I suspect that I'll have to export the files inside a .package from S3PE, and create my own text parser. Yeugh.
Advertisement
MakeNeedsStaticOn
staff: senior moderator
#2 Old 30th Jun 2017 at 7:22 PM
From as far as I understand what you're trying to do, maybe this thread might come in handy for you: http://modthesims.info/showthread.p...6&goto=lastpost
Test Subject
Original Poster
#3 Old 15th Jul 2017 at 3:05 AM
Quote:
Originally Posted by Greenplumbbob
From as far as I understand what you're trying to do, maybe this thread might come in handy for you: http://modthesims.info/showthread.p...6&goto=lastpost


Sounds similar, but the issue is much simpler; the GEOM records are pointing to duplicate _IMG (.dds) records. It should be fairly easy to resolve this by hand, through a mere text search & replace. The problem is that there are waaaaay too many mods that have these duplicated textures. Especially when merging. All those blank, 100% transparent textures...

Take this clothing mod as an example: http://www94.zippyshare.com/v/ztmukNgj/file.html There are 16 _IMG records in total. The duplicates can be sorted into the following groups, with their Instance ID's listed:

Greyscale + Transparent Background - We'll call it GTB
1. 0x4B214044741F5A67
2. 0x2CA145391012D1A1
3. 0x1D68E8893F2B44ED
4. 0x044D08966209C0B3
5. 0x1AD3F3C771921FE8
6. 0x5A3B582270A975F8
7. 0x69ED689B2DB7CFB1

Outline + Transparent Background - We'll call it OTB
1. 0x17B9586E241C2912
2. 0x587FE9D30C2C1EA0
3. 0x5C4C7CB8798D9394
4. 0x3308DDB9119D26DA
5. 0x205C1A584F87A12A
6. 0x641085376D030EA5

These instances are referenced directly in the GEOM records. I'll compare the first two in the .package.

0x0CB5A5ED47401CF7
Code:

      --- ShaderDataList: SData (0xB) ---
         [0]: Field: 0xDAA9532D (index_of_refraction); Data: 1.0000
         [1]: Field: 0x29BCDD1F (reflectivity); Data: 0.5000
         [2]: Field: 0x6E56548A (NormalMap); Index: 0x00000000 (0x00B2D882-0x00000000-0x17B9586E241C2912) <-- OTB #1
         [3]: Field: 0x73C9923E (Reflective); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [4]: Field: 0x6CC0FD85 (DiffuseMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x2CA145391012D1A1) <-- GTB #2
         [5]: Field: 0xF755F7FF (Shininess); Data: 20.0000
         [6]: Field: 0xC3FAAC4F (AlphaMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x2CA145391012D1A1) <-- GTB #2
         [7]: Field: 0x05D22FD3 (Transparency); Data: 1.0000
         [8]: Field: 0x3BD441A0 (Emission); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [9]: Field: 0x04A5DAA3 (Ambient); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [A]: Field: 0xAD528A60 (SpecularMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x2CA145391012D1A1) <-- GTB #2
      ---



0x3A36C2E40918759F
Code:

      --- ShaderDataList: SData (0xB) ---
         [0]: Field: 0xDAA9532D (index_of_refraction); Data: 1.0000
         [1]: Field: 0x29BCDD1F (reflectivity); Data: 0.5000
         [2]: Field: 0x6E56548A (NormalMap); Index: 0x00000000 (0x00B2D882-0x00000000-0x587FE9D30C2C1EA0) <-- OTB #2
         [3]: Field: 0x73C9923E (Reflective); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [4]: Field: 0x6CC0FD85 (DiffuseMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x1D68E8893F2B44ED) <-- GTB #3
         [5]: Field: 0xF755F7FF (Shininess); Data: 20.0000
         [6]: Field: 0xC3FAAC4F (AlphaMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x1D68E8893F2B44ED) <-- GTB #3
         [7]: Field: 0x05D22FD3 (Transparency); Data: 1.0000
         [8]: Field: 0x3BD441A0 (Emission); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [9]: Field: 0x04A5DAA3 (Ambient); Data0: 0.0000; Data1: 0.0000; Data2: 0.0000
         [A]: Field: 0xAD528A60 (SpecularMap); Index: 0x00000001 (0x00B2D882-0x00000000-0x1D68E8893F2B44ED) <-- GTB#3
      ---


The problem is obvious; each mesh has it's own duplicate texture. This is bad; both should be pointing to OTB #1 & GTB #1.

While I could solve this by hand, it would take me a few minutes just for this one mod. Factor in all the other mods that have this issue (at least a hundred or so)... Yuck. I'd much rather create a program to solve this, as it would be fairly trivial to program.

Parse the plain text GEOM files for the ID's of every texture
Sorting them into specific string arrays, depending on if the texture is a specular map, normal map, diffuse map, or a transparency map.
Find the .dds files through the obtained texture IDs
Compare the .md5 of every .dds file against every other .dds file
Add "original" textures to one array, duplicates to another
When finished, search GEOM objects for IDs inside the duplicates array
Replace with element from originals array

~fin~


After poking around S3PE, I noticed there's an "Export to files" function. Do the instance IDs stay the same upon export & reimport? That would allow me to operate on the files directly rather than the .package file. Not ideal, but if S3PI is useless due to the lack of docs, and I can work on exported files that will be re-imported, it's what I've got to work with.
MakeNeedsStaticOn
staff: senior moderator
#4 Old 16th Jul 2017 at 11:34 AM
The thing that you discovered about the shaders just now is the way it's being controlled. For the engine, and the sims itself, this is a really great thing because if you'd change the shader from a transparent compatible one to a non-transparent compatible one, it would easily be ignored without any trouble. Especially since you should consider it all being 'layers' who are controlling each other in order to make, for example, the mask work. I personally would probably have coded it all a little different and the entire code itself is pretty much stuck in a concrete wall that you can't just pull out of it with your bare hands. Not saying that you can't try!

I think a thing that's also worth keeping in mind is that there is a great possibility that the CC has been made with TSRW rather than by hand. TSRW is known for changing some in-game things, like merging all the XML files together for example, which can be a pain in the butt whenever you have to do something texture related. They have this different import coding compared to most sims tools that are being used for creating CC.

To answer your 'export to files' yes. as long as if you reimport it into a package file again, and this popup with instances, groups, name, etc. shows up, you don't touch it. Then it should keep every detail including instances since that's how default replacements are done as well!
Test Subject
Original Poster
#5 Old 20th Jul 2017 at 12:58 AM
Since the ID's stay the same, I've got a way forward. Using the S3PI library would be ideal due to operating on the .package directly, but this is the next best thing.

I'll see if I can use this as a learning experience for C++ as well. Might get another language under my belt.
Test Subject
Original Poster
#6 Old 31st Jul 2017 at 11:18 PM
Finally got around to giving this a go, but I hit a snag right out the gate. S3PE Shows the data in text form, but when the .package's files are exported, everything is in raw hex. Is there a way to export/import .package records in text form? Making a parser is beyond my skillset; I can do text parsing, but parsing hex data? Hoo-boy.
Back to top