Serialization

Took the opportunity while digging everything up to replace my existing ServiceStack.Text serialization code with the new, and free, NetSerializer. This claims to be faster than ProtoBuf and can be found here http://www.codeproject.com/Articles/351538/NetSerializer-A-Fast-Simple-Serializer-for-NET and on NuGet.

It dropped in OK and given the authors caveats about it not being used for long term storage – its has no versioning internally for serialized types – its noticibly faster than ServiceStack.Text.

SharpDX 3 Pain Points

Well down the road to converting my XNA4 project to SharpDX 3 and I think I’ve overcome most of the pain points. The ones that held me up were;

1) Write a SpriteBatch replacement. I did this by downloading the existing SpriteBatch class originally in the SharpDX Toolkit (now deprecated) and converted it for my use. I use this extensively in my texture preparation pipeline.
2) Write a Texture to byte[] serializer and deserializer to store textures in my content database instead of on disk. It probably would have been easier just to revert to single disk file access, but the performance penalty of opening all those little files all the time was noticably slower than fetching them from a nice efficient database (I use SQLCE but will probably move to Raven).
3) The custom vertexes work sublty differently in DirectX11 and this looked like being a real pain, until I discovered the SharpDX.D3DCompiler.ShaderReflection namespace which allowed me to calculate the various VertexElement structures directly from the compiled shader code.
4) The Constant Buffers byte alignment rules were just painful. Took a while to get them right, and found it very helpful to use the command line Shader compiler to output its headers, which helpfully had all the byte offsets in them, and it was an easy cross check to my C# struct declarations.

I’ve got the basic landscape rendering again, and I just need to extend the re-work to all the world objects – vegetation principally, which should be a matter of “rinse and repeat” then I can get my hands on the new Geomoetry and Hull shaders to get rid of some of my large geometry buffers that could be generated from texture samples – especially grass blades.

The Road to DirectX 11

Time to take a leap. Since XNA is a generation old now, and I never did use the Pipeline or Game Engine in it, I’m going to move the entire codebase to SharpDX 3. This is a big step but will unlock the DirectX 11 pipeline to me, enabling me to use Geometry Shaders (useful for procedural generation of grass and non-model geometry), and hopefully get a speed boost.

For anyone else trying this; its a big step. XNA “holds your hand” for a lot of the way, especially its useful features for saving and loading textures, dealing with Effects etc. SharpDX 3 is also the first SharpDX release to completely deprectate their XNA-lite Toolkit namespace – there are no Models, no Effects and no handholding. Its you and the bare metal, only barely wrapped in a Managed wrapper to handle COM pointer releases.

Principal issues are

  • No support for saving and loading of textures to streams or files
  • Use of cBuffer instead of XNA global parameters
  • Use of shader fragments (individual PS, VS and GS bytecode) and no Techniques
  • Sampler States defined in C# not HLSL

This should all make the code faster because the programmer is more in control of the IO going to the GPU, for instance by being able to update a cBuffer only for the parameters you’ve actually changed, but … if thats your biggest performance rendering headache, you probably have more problems than will be solved by¬†SharpDX.

I found that with a lot of cursing (not least of which is the SharpDX.org site being 404 which is a right pain when Google still indexes it) I could overcome all this, and am gradually debugging through the various code issues raised.

More updates when I get out of the other side.