Skip to main content

Joys of XML Serialization

Love it or hate it, XML is everywhere and for data and objects, it can be extremely useful.

On one of my last projects, a colleague introduced me to the useful XSD2Code project, which creates a .Net object from an XSD. This made it easy for him to build a structure that could be compiled into code to ensure everyone followed the same structure. This is extremely valuable if someone gives you an XSD as a format to write to and you want to populate it using an object.

In a more recent project, we needed to share details from a component with another calling application via a web service. Enter XMLSerializer, the .Net equivalent of taking an object and dumping it into XML.

Dim s As XmlSerializer = New XmlSerializer(Object)
Dim w As New StringWriter()

s.Serialize(w, Object)
return w.tostring

Sounds great, right? It was for a short time, but as the object got bigger, it contained more collections and references to other objects. Eventually, the Serialize method took up 100% of CPU Usage and never finished. Of course, we couldn't find the problem right off the bat so it caused lots of grief.

(without going into too much detail showcasing my ignorance on all of the specifics, XMLSerializer uses reflection to identify all of the public properties of an object and then outputs them to an XML file - if the object has a lot of objects or collections within it, it can cause a huge drain on the whole process).

Certain posts call to use the BinaryFormatter instead - which is impossible to read but when you de-serialize it, you get the objects out at the other end.

Dim formatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()

Dim ms As New MemoryStream()

formatter.Serialize(ms, Me)

Dim sXML As String
sXML = System.Convert.ToBase64String(ms.ToArray())

But this makes the string illegible to any non-.Net applications, of which there are many - especially if you plan on building publicly accessible web services.

So I started to go through my initial object and blank out certain properties, so that the serialization would work.

What I didn't realize is I could simply tell the Serializer to ignore certain attributes. Enter the XMLIgnore attribute.

Instead of simply calling New XMLSerializer, I call a method that returns the Serializer but with certain attributes that tell it to ignore specific details.

Function GetSmartSerializer() as XMLSerializer
Dim xOver As New XmlAttributeOverrides()
Dim attrs As New XmlAttributes()

attrs = New XmlAttributes()
attrs.XmlIgnore = True
xOver.Add(GetType(ObjectClass), "MyBigCollectionThatNoOneNeedsToSee", attrs)

Dim xSer As New XmlSerializer(GetType(PRAM_Data.Session), xOver)
Return xSer

With the code above, when the application calls

Dim o as XMLSerializer = GetSmartSerializer()
Dim w As New StringWriter()

s.Serialize(w, Object)
return w.tostring

It now excludes the property "MyBigCollectionThatNoOneNeedsToSee" from the XML.

More details can be found here:
XmlAttributes.XmlIgnore Property (System.Xml.Serialization)

I'm curious though - has anyone else encountered this limitation of the XMLSerializer? What solution have you used?


Unknown said…
This comment has been removed by the author.
Unknown said…
Have you considered using the json serializer? I'm starting to switch to json for large data feeds, since they typically require about half the bandwidth of an equivalent xml feed.

A few years back I was doing some serialization with hybernate and castor in Java. When I traced the bottleneck to it's source, I found that over 90% of the time was spent expanding the string builder. The performance was so bad that it threatened to derail the whole project. I ended up passing in a stringbuilder with a preset capacity so that it would not need to be continually resized.

That being said, you might see a similar result in C# by initializing the StringWriter as follows:

StringWriter w = new StringWriter(new StringBuilder(myExpectedCapacity));
Andrew MacNeill said…
Hi Brian,

The only issue right now is that I don't control the other applications accessing the data so they are expecting XML.

However, I'll add JSON as an alternate export and see if it improves performance.

Great idea - thanks!

Popular posts from this blog

Well, that explains CodePlex...

In a move that will be sure to anger open source (or rather anti-paid software, anti-Microsoft open source)  zealots, Microsoft is planning to buy GitHub . A year ago, I mused about why Microsoft would shut down CodePlex and how the world needs competing source code repositories to be strong. I'm not the only one per this Slashdot article  : "...   people have warned about GitHub becoming as large as it did as problematic because it concentrates too much of the power to make or break the open source world in a single entity, moreso because there were valid questions about GitHubs financial viability...." - Jacques Mattheij I will be interested in seeing this play out - whether developers jump ship or not. Have all the efforts Microsoft has made in pushing towards open source be seen as genuine or will all the zealots jump ship or maybe even attack? Microsoft's comment about why they shut down CodePlex referred to how spammers were using CodePlex. Well, GitHub

Attending Southwest Fox 2019 could change your life - Find out how

Southwest Fox is coming up in October and as I do every year, I spoke with the organizers Rick , Doug and Tamar on the FoxShow. Deadlines for Southwest Fox: Super-saver price (before July 1): $695 Early-bird price (before August 1): $770 Regular price (August 1 and later): $820 This year, I took a different approach with separate shows for each organizer but the main message is still the same : July 1st is their Go/No-Go date. Conferences don't talk about this very often. I don't think developers really question if Apple will hold their WWDC in June or Microsoft will hold their Build conference - but that's because those conferences are vendor-led. Southwest Fox is a community-driven conference - it's not driven by a company with an agenda. Listen to the interviews and you can hear how important each of the organizers feel the live connection between speakers and among attendees.

Virtual FoxFest - A New Way to Conference

If you haven't been keeping up with the news around the Fox community, the Southwest Fox conference has gone digital now showing up as  Virtual FoxFest .  At $49, it's a steal and a great way to learn some new ideas and get inspired. While the reasoning for this change is fairly obvious with the year of COVID - for me, this is something that has been a long time coming. I appreciate many people's needs for a physical conference but the world is very large and it's difficult to get people from around the world into a single physical location. I recently attended a single-track conference via YouTube (a Quasar conference). YouTube's Live stream provided a very handy way to watch, rewind and communicate with people online. While Tamar, Doug and Rick are still making decisions related to the streaming platform, there are lots of great options available. I'm really looking forward to it. The FoxPro community has also really felt its international roots