It has been awhile since my last post. Initially, I was looking into getting pesta working on Windows Azure. While at it, I thought I might as well upgrade it to support OpenSocial 0.9. And then while upgrading it, I looked into simplifying the serialization and deserialization of the various OpenSocial data representations for the REST API.
Previously, the serialization process to and from JSON, XML and ATOM was ported directly from Java and is based on reflection. I am currently migrating this to use libraries from ASP .NET Windows Communication Foundation, i.e. the System.Runtime.Serialization namespace. It turns out using Microsoft's WCF framework is not as straightforward as it seems. It's all fine and great if you're creating your own web service from scratch. But if you're trying to use it to fit your implementation with another existing implementation on another plaform, ie. via the XSD schema specification, you are going to run into issues.
This article attempts to show my use of the various serialization and deserialization methods just to comply to the format specified by the OpenSocial 0.9 REST API XML schema.
For a quick introduction on using ASP .NET WCF to create web services quickly and painlessly, have a look here.
The following is a quick breakdown of the advantages and disadvantages of methods for serialization and deserialization of data.
DataContractSerializer, DataContractJsonSerializer
| Advantages |
- You just need to add DataContractAttributes and DataContractMemberAttributes to mark properties that you want serialized.
- You can also use the IDataContractSurrogate interface to customised the serialization process somewhat.
- Quick and fast to implement
- You can choose whether to serialize or not if the property or field is NULL
- Serializes to both JSON, XML and ATOM quickly using the same defined attributes
|
| Disadvantages |
- Difficult to shape the output if the XSD schema does not meet the requirements for a DataContract
- Difficult to shape the output if the JSON and XML output does not really map directly
|
JavaScriptSerializer
| Advantages |
- Serialises anything and everything to JSON
- You can create JavaScriptConverters or use the ISerializable interface to customize the serialization of particular custom types
|
| Disadvantages |
- Will serialize NULL values as well
- only handles JSON
|
XmlSerializer
| Advantages |
- Serialises anything and everything to XML
- You can use the IXmlSerializable interface to customize the serialization of particular custom types
|
| Disadvantages |
|
Other issues that you will come across are as follows
- You cannot use ISerializable or IXmlSerializable in a class using DataContracts
- DataContractSerializer and DataContractJsonSerializer are not able to handle IDictionary by default
- The JavaScriptSerializer will serialize enums into integer values by default
- XmlSerializer will generate default xml namespace references
- XmlSerializer will include fields and properties with NULL values
- Deserializing all the customized output that you have generated
The final solution
The easiest solution that I found that allowed me to serialize and deserialize was achieved by the following
- Mark classes that needs to be serialized to JSON using DataContracts and DataMember attributes
- Use EmitDefaultValue to specify NULL properties that need to be excluded when serializing
- Mark classes that needs to be serialized to XML using the XmlRootAttribute
- Mark properties that needs to be excluded using XmlIgnoreAttribute and the xxxSpecified property
- The above should handle most cases, if custom XML serialization is required the IXmlSerializable interface was used
For serialization to JSON, use the JavaScriptSerializer and specify a JavaScriptConverter. This JavaScriptConverter is for serializing the classes that have been marked with DataContracts. The class is as follows
1 public class DataContractJSConverter : JavaScriptConverter
2 {
3 public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
4 {
5 throw new NotImplementedException("Unable to deserialize " + type.Name);
6 }
7
8 public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
9 {
10 var type = obj.GetType();
11 var properties = type.GetProperties();
12 var result = new Dictionary<string, object>();
13 foreach (var info in properties)
14 {
15 var value = info.GetValue(obj, null);
16 if (info.IsDefined(typeof(DataMemberAttribute), false) && value != null)
17 {
18 var valueType = value.GetType();
19 // following needed, otherwise enums are serialised into numbers
20 if (valueType.IsEnum)
21 {
22 value = value.ToString();
23 }
24 // handle scenario where only one entry
25 else if (typeof(IRestfulCollection).IsAssignableFrom(type) &&
26 typeof(IList).IsAssignableFrom(valueType))
27 {
28 var entry = ((IList) value);
29
30 if (entry.Count == 1)
31 {
32 result.Add(info.Name, entry[0]);
33 continue;
34 }
35 }
36 result.Add(info.Name, value);
37 }
38 }
39 return result;
40 }
41
42 public override IEnumerable<Type> SupportedTypes
43 {
44 get
45 {
46 return new Type[]
47 {
48 typeof(Person),
49 typeof(Account),
50 typeof(Activity),
51 typeof(Address),
52 typeof(BodyType),
53 typeof(ListField),
54 typeof(MediaItem),
55 typeof(Message),
56 typeof(MessageCollection),
57 typeof(Name),
58 typeof(Organization),
59 typeof(Url),
60 typeof(RestfulCollection<Person>),
61 typeof(RestfulCollection<Activity>),
62 };
63 }
64 }
65 }
Basically, what this JavaScriptConverter does is that it serializes only the defined types and only properties that are marked with the DataMemberAttribute and aren't NULL. The enum issue is also handled here. Note that by using the JavaScriptSerializer, it will be able to handle any IDictionary types that it comes across. For the deserialization process, the DataContractJsonSerializer is used with a surrogate for handling Dictionary types. This works out pretty well. The deserialization method and surrogate are as follows
1 public override T ConvertToObject<T>(String json)
2 {
3 DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof (T),new[]{typeof(JsonSurrogate.SDictionary)},int.MaxValue,true,new JsonSurrogate(),false);
4 MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(json));
5 var obj = serializer.ReadObject(ms);
6 return (T)obj;
7 }
The surrogate:
1 public class JsonSurrogate : IDataContractSurrogate
2 {
3 private static List<Type> knownTypes = new List<Type>
4 {
5 typeof(Dictionary<string,string>)
6 };
7 [Serializable]
8 public class SDictionary : ISerializable
9 {
10 public Dictionary<string, string> dict;
11 public SDictionary()
12 {
13 dict = new Dictionary<string, string>();
14 }
15 public SDictionary(Dictionary<string, string> dict)
16 {
17 this.dict = dict;
18 }
19
20 // deserialize
21 protected SDictionary(SerializationInfo info, StreamingContext context)
22 {
23 dict = new Dictionary<string, string>();
24 foreach (var entry in info)
25 {
26 dict.Add(entry.Name, entry.Value.ToString());
27 }
28 }
29 // serialize
30 public void GetObjectData(SerializationInfo info, StreamingContext context)
31 {
32 foreach (string key in dict.Keys)
33 {
34 info.AddValue(key, dict[key]);
35 }
36 }
37 }
38
39 private static bool IsKnownType(Type type)
40 {
41 if (knownTypes.Contains(type))
42 {
43 return true;
44 }
45 return false;
46 }
47
48 public Type GetDataContractType(Type type)
49 {
50 if (IsKnownType(type))
51 {
52 return typeof(SDictionary);
53 }
54
55 return type;
56 }
57 public object GetObjectToSerialize(object obj, Type targetType)
58 {
59 throw new NotImplementedException();
60 }
61
62 public object GetDeserializedObject(object obj, Type targetType)
63 {
64 if (obj is SDictionary)
65 {
66 return ((SDictionary) obj).dict;
67 }
68 return obj;
69 }
70
71 public Type GetReferencedTypeOnImport(string typeName, string typeNamespace, object customData)
72 {
73 return null;
74 }
75
76 #region not implemented
77
78 public object GetCustomDataToExport(MemberInfo memberInfo, Type dataContractType)
79 {
80 throw new NotImplementedException();
81 }
82
83 public object GetCustomDataToExport(Type clrType, Type dataContractType)
84 {
85 throw new NotImplementedException();
86 }
87
88 public void GetKnownCustomDataTypes(Collection<Type> customDataTypes)
89 {
90 throw new NotImplementedException();
91 }
92
93 public CodeTypeDeclaration ProcessImportedType(CodeTypeDeclaration typeDeclaration, CodeCompileUnit compileUnit)
94 {
95 throw new NotImplementedException();
96 }
97
98 #endregion
99 }
For XML serialization, the XMLSerializer is used. For removing namespaces, that are generated, ie.
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
the XmlSerializerNamespaces class is employed. The method to serialize to XML looks like the following
1 protected String convertToXml(Object obj, RequestItem request)
2 {
3 MemoryStream ms = new MemoryStream();
4 XmlWriter xw = XmlWriter.Create(ms, new XmlWriterSettings { Indent = true });
5 using (var writer = XmlDictionaryWriter.CreateDictionaryWriter(xw))
6 {
7 if (obj.GetType().IsAssignableFrom(typeof(RestfulCollection<>)))
8 {
9 IRestfulCollection collection = (IRestfulCollection)obj;
10 var responses = collection.getEntry();
11 writer.WriteStartElement("response", osNameSpace);
12 writer.WriteElementString("startIndex", collection.startIndex.ToString());
13 if (request.getCount().HasValue)
14 {
15 writer.WriteElementString("itemsPerPage", request.getCount().ToString());
16 }
17 writer.WriteElementString("totalResults", collection.totalResults.ToString());
18 writer.WriteElementString("isFiltered", collection.isFiltered.ToString().ToLower());
19 writer.WriteElementString("isSorted", collection.isSorted.ToString().ToLower());
20 writer.WriteElementString("isUpdatedSince", collection.isUpdatedSince.ToString().ToLower());
21
22 foreach (var item in (IList)responses)
23 {
24 writer.WriteStartElement("entry");
25 //ser.WriteObject(writer, item);
26 XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
27 ns.Add("", osNameSpace);
28 XmlSerializer ser = new XmlSerializer(item.GetType());
29 ser.Serialize(writer, item, ns);
30 writer.WriteEndElement();
31 }
32
33 writer.WriteEndElement();
34 }
35 else if (obj is DataCollection)
36 {
37 XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
38 ns.Add("", osNameSpace);
39 XmlSerializer ser = new XmlSerializer(obj.GetType());
40 ser.Serialize(writer, obj, ns);
41 }
42 else
43 {
44 throw new Exception("XMLSerialization: shouldn't have got here");
45 }
46 writer.Flush();
47 return Encoding.UTF8.GetString(ms.ToArray());
48 }
49 }
Atom serialization is achieved via the XMLSerializer as well. And had to be manually formatted to fit the OpenSocial XML schema. Details of the converter will be delivered to the archive soon.