Stream

From Niotso Wiki
Jump to: navigation, search
Stream
Type of format Compression
Byte order Variable
Container for RefPack-compressed data, uncompressed data
Contained by FAR
cTSOSerializableStream (which, for brevity, this article will abbreviate as Stream) is a format used in The Sims Online that contains data, either uncompressed or compressed with RefPack. In The Sims Online, Stream files can be found standalone or within the game's FARv3 files. To put it into perspective, the Stream format is similar to gzip (i.e. a container for either uncompressed or compressed data), RefPack is similar to DEFLATE (i.e. a compression algorithm), and FARv3 is similar to tar (i.e. a file containing multiple files).

Some examples of standalone Stream files are tuning.dat, censor/censor.dat, and the extensionless files in userdata/localavatarcache produced after the player enters the city.

Stream files can have either a little-endian header or a big-endian header. When writing to the disk, the game always uses little-endian format and type=1. When sending over the network, the game generally uses big-endian and type=3, although one exception exists: the cTSOAvatarCreationRequest that the client sends to the server in CAS is a Stream file containing another Stream file; the outer Stream (type 3) is big-endian, while the inside Stream (also type 3) is little-endian.

The format is as follows:

  • Body type - A single byte specifying the format the body follows; should be 0, 1, or 3. 0 means uncompressed, while 1 and 3 both mean compressed. Currently, the difference between 1 and 3 is not well understood; it is guessed that 3 means "higher compression has been applied" compared to 1, or vice versa, or that 3 means "probably big-endian" and 1 means "probably little-endian".
  • Decompressed data size - A 4-byte unsigned integer in the current channel's byte order specifying the size of the data when decompressed

If Body type is zero:

  • Data - Raw, uncompressed data

Otherwise, if Body type is nonzero:

  • Compressed data size - A 4-byte unsigned integer in the current channel's byte order specifying the size of the body which follows these 9 header bytes
  • Body:
    • Body size - A 4-byte unsigned integer in little-endian order (regardless of the channel) specifying the size of this body; should be equal to the Compressed data size
    • Compressed data - Compressed data compressed with RefPack

Naming notes[edit]

In EA-Land, the function that reads Stream files is located at TSOServiceClientD_base+0x817e0. According to the RTTI in EA-Land's version of TSOServiceClientD.dll, the thisptr in ecx points to an object of type "cTSOSerializableStream". The QueryInterface function (the first function in the v-table of this object) is located at TSOServiceClientD_base+0x81a90, and it accepts the following IIDs (names provided by the Pre-Alpha constants table):

  • 0x0898CFA8 (GZIID_cITSOSerializableStream)
  • 0x853D26DE (GZIID_cIGZSerializableStream)
  • 0xE4FDA3D4 (GZIID_cIGZSerializable)
  • 0xC51B11CB (GZIID_cIGZOStream)
  • 0x00000001 (IUnknown)

Additionally, this v-table is associated with the clsid 0xDBB9126C (GZCLSID_cTSOSerializableStream) at TSOServiceClientD_base+0x304c5 in EA-Land.

Additionally, in TSO New & Improved Trial, the function that reads Stream files is located at TSOServiceClientD_base+0x6704d. The thisptr in ecx also points to an object whose QueryInterface function accepts the same IIDs as above and whose v-table is also associated with 0xDBB9126C at TSOServiceClientD_base+0xcd58.

Additionally, all versions of the TSO client use the 0xDBB9126C clsid (GZCLSID_cTSOSerializableStream) when sending the cTSOAvatarCreationRequest packet to the server in CAS.