Why Has the Size of TObject Doubled In Delphi 2009?
Mason Wheeler noticed that TObject.InstanceSize returns 8 (bytes). It turns out that this is new in Delphi 2009; in previous releases, TObject.InstanceSize returned 4. But when you look at the definition of TObject in System.pas, you don't see any fields declared at all.
Four out of the eight bytes are consumed by the VMT; this has been true since the first version of Delphi. You can read more about that in this chapter from Delphi In A Nutshell, which is old, but still accurate insofar as the VMT is concerned.
But what about the remaining four bytes? I didn't remember the answer, but I enjoy a good puzzle, so I took a closer look at the source code for TObject. While poking around in System.pas, I happened to notice the following constants:
{ Hidden TObject field info }
hfFieldSize = 4;
hfMonitorOffset = 0;
OK, that explains the four bytes, but what is this used for? Searching for
hfFieldSize yielded the answer:
class function TMonitor.GetFieldAddress(AObject: TObject): PPMonitor;
begin
Result := PPMonitor(Integer(AObject) + AObject.InstanceSize - hfFieldSize + hfMonitorOffset);
end;
So any object can have a reference to a TMonitor instance, and that reference is as big as a pointer on the target CPU. That reminded me of Allen Bauer's long series of blog posts last year on the "Delphi Parallel Library," which discuss the TMonitor class in detail. He notes:
The TMonitor class [...] is now tied directly to any TObject instance or derivative (which means any instance of a Delphi class). [...] For Tiburón [a.k.a. Delphi 2009], you merely need to call System.TMonitor.Enter(<obj>); or System.TMonitor.Exit(<obj>); among the other related methods.
The ability to lock any object is a good feature to have, especially in a multi-CPU, multi-core world. So I'm happy to spend the extra four bytes to get this feature. But I do find the implementation a bit mysterious. Why not just declare an actual field in TObject? I guess one reason would be to reduce the chance of a conflict with existing code. Another possible reason would be to enforce TMonitor's contract; by obfuscating access to the field, you reduce the chances that an uninformed programmer performs actions on the field which break the contract. But these are just guesses. I don't know why the Delphi team settled on this implementation. I presume they have some good reason.
Update: In comments, Allen Bauer explains the reasons for this implementation:
The reason for always placing the monitor reference field at the end of the object instance is mainly for backward compatibility. I didn’t want to alter the layout of objects in case there was some code out there that depended upon it. Also, by hiding the implementation the chance of inadvertent mucking with the monitor data was reduced and therefore it increased the overall safety of using it.
If you look closely at what happens on descendant instances, the monitor field is always the last field of the instance. This happens because the compilers (Delphi & C++) ensure that the field is allocated as the last step in the process of laying out the class instance. By doing this, an object laid out in pre-Delphi 2009 will be laid out exactly the same in Delphi 2009 except there is now an extra trailing pointer-sized field.
Comments
-
Wednesday, 25 March 2009The reason for always placing the monitor reference field at the end of the object instance is mainly for backward compatibility. I didn't want to alter the layout of objects in case there was some code out there that depended upon it. Also, by hiding the implementation the chance of inadvertent mucking with the monitor data was reduced and therefore it increased the overall safety of using it.
If you look closely at what happens on descendant instances, the monitor field is *always* the last field of the instance. This happens because the compilers (Delphi & C++) ensure that the field is allocated as the last step in the process of laying out the class instance. By doing this, an object laid out in pre-Delphi 2009 will be laid out exactly the same in Delphi 2009 except there is now an extra trailing pointer-sized field. -
Wednesday, 25 March 2009
Not every object has the lock. The actual lock itself is a record that is larger than SizeOf(POinter) bytes. That extra field is merely a *reference* to the lock and only occupies SizeOf(Pointer) bytes. The monitor itself is allocated on demand when you begin to use the System.TMonitor methods. By using this indirection, in future releases this could also open things up to adding more run-time allocated "meta" information. Things such as attaching dynamic marshaling helpers, run-time attributing, and other interesting things.
-
Wednesday, 25 March 2009
So, what your saying is that if I created a NEW record structure, which included as the first part of its structure TMonitor, I could in thory add any metadata I so desired linked to an object instance? So something like the following would be possible:
TMyInstanceMetaData = record
MonitorCompatRecord : TMonitor;
MyMetaData : String;
end;
then creating a new GetMonitor class function which would allocate and link the new record. -
Wednesday, 25 March 2009
Ah, so its back to using a tDictionary to store the metadata then... at least as long as one doesn't for..each in another unit (hoping the next patch fixes that issue)
-
Wednesday, 25 March 2009
and that was tDictionary<TObject,TMetaDataRec> but I forgot I was commenting in HTML.
-
Wednesday, 25 March 2009
*laughs* TDictionary is so broken. I agree; I'd like to see it fixed RSN.
Thanks for the link, Craig. I just hope I don't get a bunch of people wandering in, glancing around for a second, saying "hey, this isn't a programming blog!" and then leaving again. :P -
Wednesday, 25 March 2009
Yeah. I was at a Delphi Users' Group meeting about 3 weeks ago, where Anders Ohlsson said it would be out "any day now."
Oh well. If they're going to delay things, I really hope they can find time to include fixes for QC #s 72070 and 71837 in there. Bad VCL code or strange generics quirks that don't always compile right are bad enough, but when the compiler silently creates a bad binary, that's a critical error IMO. -
Wednesday, 25 March 2009
Just for the record, it was Thorsten Engler who mentioned this extension first (he wrote about this already on july 22, 2008).
I didn't realize it at that time, but it never occurred to me Allan's TMonitor experiments where meant to be actually released with Delphi 2009. Only nowI discover InstanceSize changed to 8 bytes (thanks Mason), without a mention anywhere (or was it?)!
Am I the only one getting freaked out about this?!
Our application allocates literally hundreds of millions of instances... if it wasn't for our own instance-pool (which works with fixed instance sizes), this would have meant a killing increase in memory-usage!
Surely others must get EOutOfMemory too now that they start releasing Delphi 2009 builds of there software?!?
I wish this was communicated more openly, and a fallback to the original InstanceSize was kept possible (because seriously, how many of you are already using the new TMonitor design?) -
Thursday, 26 March 2009
Yeah, 4 bytes is not much, especially in relation to already-large instances. And I do agree on all your points.
But still, the virtual-memory limit on win32 will now be reached sooner than before (even more so now that string has become UTF16 encoded). Not many applications will notice this, sure. But some will. That's why I would have expected that a change like this had been communicated more clearly.
Also, I don't see why this was actually released together with Tiburon. There's no documentation or mention of the new TMonitor functionality either (unless I've somehow missed this?)
And on top of all that, there's no feasible way to get back the original InstanceSize!
I already discussed some options with Hallvard Vassbotn, like patching TObject.GetInstanceSize (which won't work, as that's marked 'inline' now) or patching TObject.NewInstance (which could work - allocation-wise, but would still report the wrong InstanceSize). In any case, we would have to nullify TMonitor.DestroyObject (or what's-it's-name). Last resort would be to detect all classes early in the initialization phase, and decrease all vmtInstanceSize values by 4. None of this is a 'clean' 'fix', so it would have been nicer if this increase was either left out, or was done via a project-option or some such.
For now, I guess most people will probably accept this without much objections. (I suspect I too won't even bother patching this.) But I do hope that next time (if you can speak of a 'next time' for things like this) we'd be informed beforehand, and where offered a choice. (Actually, the Unicode-move is a great example, as we can still use AnsiString if we wish to do so). -
Thursday, 26 March 2009
Patrick:
Take a look at Steven Kamradt's idea a few posts up. If CodeGear does something like this, there shouldn't ever have to be a "next time". I hope. -
Please login first in order for you to submit comments
- Page :
- 1
Mason was asking me why every object needed the ability to lock. I suggested that with multi-core code and the possibility of a parallel for loop or future then that sort of behavior is really important and will be used frequently.