ObjectManager Problem - Plea for help!

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I'm trying to write a custom formatter. I'm using the .NET Framework's
ObjectManager class to do so, but I can't persuade the it to fixup object
references in a value type.

Some cut-down code that reproduces the problem I'm having is shown below.

Can anyone help me? (Is there anyone from Microsoft that can pass this to
someone who is familiar with the ObjectManager? Please?)

Thanks,
- Lee


using System;
using System.Runtime.Serialization;

namespace Test
{
class C1
{
static void Main()
{
// Pretend we're deserializing from a stream. We're aiming for:
//
// C2 root = new C2();
// root.m_s = new S();
// root.m_s.m_i = 100;
// root.m_s.m_o = new C3();
// root.m_s.m_o.m_i = 4;

// c2_info will be used to deserialize an instance of C2.

SerializationInfo c2_info = new SerializationInfo(typeof(C2), new
FormatterConverter());
S s = new S();
s.m_i = 100;
c2_info.AddValue("m_s", s); // Box the value type, s.

// c3_info will be used to deserialize an instance of C3

SerializationInfo c3_info = new SerializationInfo(typeof(C3), new
FormatterConverter());
c3_info.AddValue("m_i", 4);

// Build the object graph we're aming for using the
// SerializationInfo objects.

ObjectManager om = new ObjectManager(null, new
StreamingContext(StreamingContextStates.All, null));

// Create an empty C2 and register it.

object c2 = FormatterServices.GetUninitializedObject(typeof(C2));
om.RegisterObject(c2, 1, c2_info);

// Create an empty S and register it.

object boxed_s = c2_info.GetValue("m_s", typeof(S));
om.RegisterObject(boxed_s, 2, null, 1, null);
// I don't think the last parameter should be null, but I
// don't know what to set it to, baring in mind that C2
// implements ISerializable.

// Ask the ObjectManager to perform 'boxed_s.m_o = c3'
// assignment during fixup.

om.RecordFixup(2, typeof(S).GetField("m_o"), 3);

// Create an empty C3 and register it.

object c3 = FormatterServices.GetUninitializedObject(typeof(C3));
om.RegisterObject(c3, 3, c3_info);

Console.WriteLine("m_i = {0}, m_o = {1}", ((S)boxed_s).m_i,
((S)boxed_s).m_o == null ? "null" : "NOT null");
// m_i = 100, m_o = null
// This is correct: the fixup hasn't happend yet.

om.DoFixups();

Console.WriteLine("m_i = {0}, m_o = {1}", ((S)boxed_s).m_i,
((S)boxed_s).m_o == null ? "null" : "NOT null");
// m_i = 100, m_o = NOT null
// This is correct (or at least promising): the
// fixup has happened.

C2 root = (C2)om.GetObject(1);

Console.WriteLine("m_i = {0}, m_o = {1}", root.m_s.m_i, root.m_s.m_o ==
null ? "null" : "NOT null");
// m_i = 100, m_o = null
// But the fixup hasn't made it to the value type
// in the C2 instance registered as object #1.

// C2 is being completed before the boxed_s.m_o fixup,
// so the value copy in C2() is copying the unfixed-up object!
// What am I doing wrong!?

Console.ReadLine();
}
}

[Serializable]
class C2 : ISerializable
{
public S m_s;

C2(SerializationInfo info, StreamingContext context)
{
m_s = (S)info.GetValue("m_s", typeof(S));
}

public void GetObjectData(SerializationInfo info, StreamingContext
context)
{
// We don't need this as we're only deserializing, not serializing.
}
}

[Serializable]
struct S
{
public int m_i;
public object m_o;
}

[Serializable]
class C3 : ISerializable
{
public int m_i;

#region ISerializable Members

private C3(SerializationInfo info, StreamingContext context)
{
m_i = info.GetInt32("m_i");
}

public void GetObjectData(SerializationInfo info, StreamingContext
context)
{
// We don't need this as we're only deserializing, not serializing.
}

#endregion
}
}
 
Hi Lee,

We have reviewed this issue and are currently researching on it. We will
update you ASAP. Thanks for your patience!

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."
 
Hi Lee,

Thanks for your posting.Regarding on your description, seems you're using
the ObjectManager class to manually do something which should be done
interanlly via the .net provided formatter class, yes?

Would you provide some further description on the reason why you want to do
the manual formatter since the buildin formatters have provided the most
sufficient functions. Since I'm also not very familiar on directly use the
objectmanager to construct the serializationinfo, I may need some further
info on your requirement and the context info so that I can try looking for
some other experts who may help on this.

Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Steven Cheng said:
Hi Lee,

Thanks for your posting.Regarding on your description, seems you're using
the ObjectManager class to manually do something which should be done
interanlly via the .net provided formatter class, yes?
Click to expand...

Yes. I'm writing a custom formatter. i.e. I'm inheriting from
System.Runtime.Serialization.Formatter and implementing the IFormatter
interface. I am trying to utilise the ObjectManager class that has been made
available for this purpose. My actual code is a lot more complex that the
example I posted, but the simplified example does replicate the problem I
have encountered.
Would you provide some further description on the reason why you want to do
the manual formatter since the buildin formatters have provided the most
sufficient functions. Since I'm also not very familiar on directly use the
objectmanager to construct the serializationinfo, I may need some further
info on your requirement and the context info so that I can try looking for
some other experts who may help on this.
Click to expand...

The company I am working for is developing a tool. You can think of this
tool as being similar to Microsoft Visual Studio, but only in the sense that
it is an editor that needs to be able to parse and produce a number of
different file formats.

We have a number of classes and structures that represent the objects in our
problem domain, and I hope to use serialization to solve two specific
problems. The first is to produce files in these different file formats; this
is the main purpose of the tool. The second is to store the same data in a
single file so that it can be archived and transfered between physical
servers, including via email.

If each object knows how to serialize itself, then I can use the standard
BinaryFormatter and FileStream classes for archiving, and I plan to use
custom formatter and stream objects to produce the specific file formats.
That way, when I develop a new file format, I get the archiving for free.

I hope this provides you with the information you were after.
 
Hi Lee,

Thanks for your followup and the further description. I'll forward your
request and the code snippet to our CLR guys for some values. I'll update
you as soon as I got any new infos.
Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Hi Lee,

After some further consulting, our dev guys found that the following change
can do the work:

replacing

============
object boxed_s = c2_info.GetValue("m_s", typeof(S));

om.RegisterObject(boxed_s, 2, null, 1, null);
============

with

==============
object boxed_s = c2_info.GetValue("m_s", typeof(S));

System.Reflection.MemberInfo[] m =typeof(C2).GetMember("m_s");

om.RegisterObject(boxed_s, 2, null, 1, m[0]);
om.DoFixups();
==============

Since the boxed_s is member (m_s) of c2 , we need to specify the memberinfo
in the 4th paramter

Also, after the change, you'll found that your second test statement
=====================
((S)boxed_s).m_o == null
=====================

will always return true which means the boxed_s 's m_o is null. And the dev
guy think this was because once we specify S (objid 2) as a child of
C2(Objid 1) then any Fixup on the S is directly happening on C2¡¯s child S
and the boxed_s 's "m_o" won't be set to c3(object id =3).

In addition, it seems that there hasn't been any detailed reference or
documtation on manually deserializing via the ObjectManager class since its
internal mechanism is hidden from us.

Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)






thanks,

liju
 
Thanks very much for helping with this, but, unfortunately, I don't think
we're quite there yet.
After some further consulting, our dev guys found that the following change
can do the work:

replacing

============
object boxed_s = c2_info.GetValue("m_s", typeof(S));

om.RegisterObject(boxed_s, 2, null, 1, null);
============

with

==============
object boxed_s = c2_info.GetValue("m_s", typeof(S));

System.Reflection.MemberInfo[] m =typeof(C2).GetMember("m_s");

om.RegisterObject(boxed_s, 2, null, 1, m[0]);
om.DoFixups();
==============

Since the boxed_s is member (m_s) of c2 , we need to specify the memberinfo
in the 4th paramter
Click to expand...

I think your dev guys may have missed the point I was trying to make in the
C# comments. I didn't think that the 4th parameter to RegisterObject() was
supposed to be null, but I don't think it can be an expression based on the
name of one of C2's fields. This is because C2 implements ISerializable. Let
me expand upon that.

You are suggesting that, because S is value-type (a struct), when I register
an instance of S with the ObjectManager, the 4th parameter to
RegisterObject() should be - as the documentation for RegisterObject says -
"the field in the containing object where S exists". To do this I can, as you
have suggested, try to use Type.GetMember(), or even Type.GetField():

om.RegisterObject(boxed_s, 2, null, 1, typeof(C2).GetField("m_s"));

The problem is that during deserialization I don't know that C2.m_s is the
containing field. I know that boxed_s belongs in a specific instance of C2,
yes, but I don't know that it is the m_s field. As I've said, this is because
C2 implements ISerializable: the GetObjectData() method and the custom
constructor required for custom serialization allows there to be a level of
indirection between an object's field and the name that field's data is
serialized under. For example:

[Serializable]
class C2 : ISerializable
{
public S m_s;

C2(SerializationInfo info, StreamingContext context)
{
m_s = (S)info.GetValue("ArbitraryName", typeof(S));
}

public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("ArbitraryName", m_s);
}
}

So now, the formatter code would need to change to:

object boxed_s = c2_info.GetValue("ArbitraryName", typeof(S));
om.RegisterObject(boxed_s, 2, null, 1, typeof(C2).GetField("m_s"));

But where do I get the string "m_s" from? As far as the formatter knows,
this field is known as "ArbitraryName", not the actual field name, "m_s".
(During deserialization of an instance of C2, the formatter could enumerate
the SerializationInfo object extracted from the underlying stream via
SerializationInfo.GetEnumerator(), but it would find the name
"ArbitraryName", not "m_s".) As far as I can see, the only link between these
two strings is in the code for ISerializable.GetObjectData() and the custom
constructor, which I - as the author of the formatter, not the objects it
deserializes - have no control over.

Does that make sense?

The second problem I have is with calling DoFixups() more than once. The
solution you are proposing involves calling DoFixups() as soon as boxed_s is
registered with the ObjectManager, but I can't guarantee that at this point
in time the ObjectManager will have had all referenced objects registered
with it.

For example, if we ignore the m_s problem for a moment and I record a fixup
on a totally different object before I even encounter C2 then the code
becomes:

om.RegisterObject(new C4(), 4);
om.RecordFixup(4, typeof(C4).GetField("whatever"), 5);

object boxed_s = c2_info.GetValue("m_s", typeof(S));
om.RegisterObject(boxed_s, 2, null, 1, typeof(C2).GetField("m_s"));
om.DoFixups();

And the call to DoFixups() generates an exception: "The object with ID 5 was
referenced in a fixup but does not exist."

For completeness, C4 is defined as:

[Serializable]
class C4
{
public object whatever;
}

and the deserialization stream would eventually yield:

om.RegisterObject(new C4(), 5);

(I appreciate that in this example, there is no explicit link between the
instances of C4 and the other objects in the graph, but there easily could
be: C2 could have another field, m_c4.)
Also, after the change, you'll found that your second test statement
=====================
((S)boxed_s).m_o == null
=====================

will always return true which means the boxed_s 's m_o is null. And the dev
guy think this was because once we specify S (objid 2) as a child of
C2(Objid 1) then any Fixup on the S is directly happening on C2¡¯s child S
and the boxed_s 's "m_o" won't be set to c3(object id =3).
Click to expand...

Yes, that makes sense - at least for containing objects that don't implement
ISerializable. I'm not sure it makes sense to me for containing objects that
do implement ISerializable.
In addition, it seems that there hasn't been any detailed reference or
documtation on manually deserializing via the ObjectManager class since its
internal mechanism is hidden from us.
Click to expand...

Doh. Okay, where do we go from here then?

Thanks,
- Lee
 
Hi Lee,

Thanks for the followup. Since I'm also not sure on the ObjectManager
class's actual behavior, I'll have to ask some other proper ones for
further assistance. And we'll keep focus on the issue and update you when
we got any new info. Thanks for your understanding.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Hi Lee,

I've reported this problem to the DEV team and some dev engineers are
currently work on this , but seems this will be a issue addressed in the
next version of the framework. Is it possible that we use class instead of
struct currently?
Please feel free to let us know any of your concerns. Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Hi,

To be honest, I'm not sure if I can always use a class instead of a struct.
As the author of the formatter, I don't necessarily have control over the
objects that it will attempt to deserialize. In this particular case, the
struct that is causing me problems is one of ours, and so I could, I guess,
make it a class instead. However, I'm concerned that we may encounter a
third-party component or possibly even a Framework component that uses a
struct in this way. I'm also worried about having to restrict the use of
structs just to get serialization to work properly. I may end up abandoning
the approach for something a little less risky.

It is somewhat disappointing that the ObjectManager has this problem. I am
particularly surprised as the built-in formatters have no problems
deserializing such objects. This implies that the built-in formatters are not
using the ObjectManager to deserialize. It is very frustrating that a class
that is not good enough for the built-in formatters has been published and
documented to the contrary. If the built-in formatters are using a different
object during deserialization, then why wasn't that object exposed instead?

I came across a similar situation a couple of years ago: I was trying to
integrate an application with Microsoft Outlook. There was an API provided
for that purpose, but it didn't work properly. Again, as the Outlook client
had no problems, it was clear that it was not using the same API.

My biggest concern is that Microsoft, as a company, is still apparently not
using the same APIs that it expects non-Microsoft developers to use. But I
don't expect we can do much about that!

Thanks for your help with this. While the outcome is frustrating, at least I
can now move on.

- Lee
 
Hi Lee,

I've got the new update from the DEV guy, seems the problem can be
resolved. Here is the modified

Main function :

================================
static void Main()
{

SerializationInfo c2_info = new SerializationInfo(typeof(C2), new
FormatterConverter());
S s = new S();
s.m_i = 100;
c2_info.AddValue("m_s", s); // Box the value type, s.

// c3_info will be used to deserialize an instance of C3

SerializationInfo c3_info = new SerializationInfo(typeof(C3), new
FormatterConverter());
c3_info.AddValue("m_i", 4);

// Build the object graph we're aming for using the
// SerializationInfo objects.

ObjectManager om = new ObjectManager(null, new
StreamingContext(StreamingContextStates.All, null));

// Create an empty C2 and register it.

object c2 = FormatterServices.GetUninitializedObject(typeof(C2));

// Create an empty S and register it.

object boxed_s = c2_info.GetValue("m_s", typeof(S));
om.RegisterObject(boxed_s, 2, null, 1, null);
// I don't think the last parameter should be null, but I
// don't know what to set it to, baring in mind that C2
// implements ISerializable.

// Ask the ObjectManager to perform 'boxed_s.m_o = c3'
// assignment during fixup.

om.RecordFixup(2, typeof(S).GetField("m_o"), 3);

// Create an empty C3 and register it.

object c3 = FormatterServices.GetUninitializedObject(typeof(C3));
om.RegisterObject(c3, 3, c3_info);
om.RegisterObject(c2, 1, c2_info);

Console.WriteLine("m_i = {0}, m_o = {1}", ((S)boxed_s).m_i,
((S)boxed_s).m_o == null ? "null" : "NOT null");
// m_i = 100, m_o = null
// This is correct: the fixup hasn't happend yet.

om.DoFixups();

Console.WriteLine("m_i = {0}, m_o = {1}", ((S)boxed_s).m_i,
((S)boxed_s).m_o == null ? "null" : "NOT null");
// m_i = 100, m_o = NOT null
// This is correct (or at least promising): the
// fixup has happened.

C2 root = (C2)om.GetObject(1);

Console.WriteLine("m_i = {0}, m_o = {1}", root.m_s.m_i, root.m_s.m_o ==
null ? "null" : "NOT null");
// m_i = 100, m_o = null
// But the fixup hasn't made it to the value type
// in the C2 instance registered as object #1.

// C2 is being completed before the boxed_s.m_o fixup,
// so the value copy in C2() is copying the unfixed-up object!
// What am I doing wrong!?

Console.ReadLine();
}
}
=========================================

And below is the dev guy's comment on the above modification:
======================
You need to register the first fixup after the others. You will notice that
is the struct is changed to a class things work just fine. Since unboxing
will loose identity you need to make sure you register the first object
last. That should fixup with struct in the boxed version and then fix it up
with the root. Attached bug.cs shows the workaround.
======================

Hope helps.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Thanks. This is looking promising.

I've had a play around, and I notice that the c2 instance (ID 1) must be
registered after the c3 instance (ID 3) for this to work. Or put another way,
the crux of the work-around appears to be that: an object that implements
ISerializable should not be registered until all of the fixups in its structs
have had *their* objects registered.

Now I'm wondering how I can implement this requirement in a generic
deserialization algorithm. My current algorithm goes:

1. Encounter an object in the stream.
2. Register the object.
3. Record the fixups.
4. Repeat until there are no more objects.

(This is obviously simplified, but you get the idea.)

Now we've identified that step 2 is incorrect, but where should it go?

I will have to look into this when I have a little more time over the next
few days. I'll let you know how I get on.

- Lee
 
Hi Lee,

Yes, from the dev guy's description, seems we need to follow the sequence
that register the container object later than the sub referenced objects.
Hope this won't make your work become too complex.

Good Luck!

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)
 
Back
Top