Skip to main content

Joys of XML Serialization

Love it or hate it, XML is everywhere and for data and objects, it can be extremely useful.

On one of my last projects, a colleague introduced me to the useful XSD2Code project, which creates a .Net object from an XSD. This made it easy for him to build a structure that could be compiled into code to ensure everyone followed the same structure. This is extremely valuable if someone gives you an XSD as a format to write to and you want to populate it using an object.

In a more recent project, we needed to share details from a component with another calling application via a web service. Enter XMLSerializer, the .Net equivalent of taking an object and dumping it into XML.

Dim s As XmlSerializer = New XmlSerializer(Object)
Dim w As New StringWriter()


s.Serialize(w, Object)
return w.tostring

Sounds great, right? It was for a short time, but as the object got bigger, it contained more collections and references to other objects. Eventually, the Serialize method took up 100% of CPU Usage and never finished. Of course, we couldn't find the problem right off the bat so it caused lots of grief.

(without going into too much detail showcasing my ignorance on all of the specifics, XMLSerializer uses reflection to identify all of the public properties of an object and then outputs them to an XML file - if the object has a lot of objects or collections within it, it can cause a huge drain on the whole process).

Certain posts call to use the BinaryFormatter instead - which is impossible to read but when you de-serialize it, you get the objects out at the other end.

Dim formatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()


Dim ms As New MemoryStream()


formatter.Serialize(ms, Me)


Dim sXML As String
sXML = System.Convert.ToBase64String(ms.ToArray())

But this makes the string illegible to any non-.Net applications, of which there are many - especially if you plan on building publicly accessible web services.

So I started to go through my initial object and blank out certain properties, so that the serialization would work.

What I didn't realize is I could simply tell the Serializer to ignore certain attributes. Enter the XMLIgnore attribute.

Instead of simply calling New XMLSerializer, I call a method that returns the Serializer but with certain attributes that tell it to ignore specific details.

Function GetSmartSerializer() as XMLSerializer
Dim xOver As New XmlAttributeOverrides()
Dim attrs As New XmlAttributes()


attrs = New XmlAttributes()
attrs.XmlIgnore = True
xOver.Add(GetType(ObjectClass), "MyBigCollectionThatNoOneNeedsToSee", attrs)


Dim xSer As New XmlSerializer(GetType(PRAM_Data.Session), xOver)
Return xSer

With the code above, when the application calls

Dim o as XMLSerializer = GetSmartSerializer()
Dim w As New StringWriter()


s.Serialize(w, Object)
return w.tostring

It now excludes the property "MyBigCollectionThatNoOneNeedsToSee" from the XML.

More details can be found here:
XmlAttributes.XmlIgnore Property (System.Xml.Serialization)

I'm curious though - has anyone else encountered this limitation of the XMLSerializer? What solution have you used?

Comments

BMarquis said…
This comment has been removed by the author.
BMarquis said…
Have you considered using the json serializer? I'm starting to switch to json for large data feeds, since they typically require about half the bandwidth of an equivalent xml feed.

A few years back I was doing some serialization with hybernate and castor in Java. When I traced the bottleneck to it's source, I found that over 90% of the time was spent expanding the string builder. The performance was so bad that it threatened to derail the whole project. I ended up passing in a stringbuilder with a preset capacity so that it would not need to be continually resized.

That being said, you might see a similar result in C# by initializing the StringWriter as follows:

StringWriter w = new StringWriter(new StringBuilder(myExpectedCapacity));
Andrew MacNeill said…
Hi Brian,

The only issue right now is that I don't control the other applications accessing the data so they are expecting XML.

However, I'll add JSON as an alternate export and see if it improves performance.

Great idea - thanks!

Popular posts from this blog

Programmers vs. Developers vs. Architects

I received an email this morning from Brandon Savage's newsletter. Brandon's a PHP guru (works at Mozilla) but his newsletter and books have some great overall perspectives for developers of all languages. However, this last one (What's the difference between developers and architects?) kind of rubs me the wrong way. Either that, or I've just missed the natural inflation of job descriptions. (maybe, it's like the change in terminology between Garbage man and Waste Engineer or Secretary and Office Administrator)

So maybe it's just me - but I think there's still a big difference between Programmer, Developer and then of course, architect. The key thing here is that every role has a different perspective and every one of those perspectives has value. The original MSF create roles like Product Manager, Program Manager, Developer, Tester, etc - so every concept may pigeon hole people into different roles. But the statements Brandon makes are often distinctions I…

Security in Windows 10

http://www.slate.com/articles/technology/bitwise/2015/08/windows_10_privacy_problems_here_s_how_bad_they_are_and_how_to_plug_them.single.html

 discusses some Windows 10 privacy settings and their implications.

"Finally, we will access, disclose and preserve personal data, including your content (such as the content of your emails, other private communications or files in private folders), when we have a good faith belief that doing so is necessary." "In other words, Microsoft won't treat your local data with any more privacy than it treats your data on its servers and may upload your local data to its servers arbitrarily"
I did a quick install on a VM choosing the Express settings. When I fully deploy this on a real workstation, I will likely choose to wade through all of the individual pages, as David recommends.

Of course, losing one's privacy is nothing new - it's happening all over the place (despite Santa Ana's police force's lawsu…

AppleSoft

I'm not TRYING to be "fanboy-flame bait" but what I saw yesterday was a typical "Do it this way, now do it this way and then we'll go back to this way" all over again.... a move similar to what Microsoft does to developers on an ongoing basis.

Remember the first iPhone? Smooth and curved, at least as far as it could be back then. I still pull out my 3G and can see the curves on it.

Then the 4 came out and "boxy" was all the rage. Everything should be "tight with corners"

Now iPhone 6.... smooth and curvy is back. Granted I don't have the actual device yet, but that's the message.

Guess that means the iPhone 8 will be back to boxy.

And honestly, Apple Watch is not worth "one more thing" --- especially when everyone knows it's going to be shown. "One more thing" would be something no one saw coming.  The device itself ? Very interesting and yes, definitely lots of potential but "one more thing" wor…