Bit Depth Confusion

edited January 2020 in ShaderMap 4

Hi Neil,

I do not understand why all maps are generated in 8 bit, regardless of the source bit-depth?

For example, if I feed a 32-bit floating point normal map I expect to get displacement in a 32-bit float. But in shadermap I will get quantized integer map even if I save it in 32-bit exr file. It is obvious that generated data is integer by typical stepping artifacts. What's the point of writing int data in 32bit float exr? Maybe I missed something in settings?

Is there any way to control map generation bit-depth (8, 16, 32 bit int/float)? Also it will be nice to have a feature to choose a format for displacement/normals -1 to 1 for float exr's and normalized 0 to 1 for integer formats (png, tif etc.).



  • Hi verno,

    All maps are generated using 32 bit floating point pixels then stored in 16 bit floating point pixels for the duration of the session. I am unsure of why stepping is occurring, let's try to find out.

    1. Are you seeing the stepping artifacts when exporting to a 16 bit format or just 32 bit? My thoughts are that this could be an issue with the exporting of 32 bit images.

    2. Is the stepping artifact only found on the "Normal -> Displacement" map or is it also found in other maps? It would be helpful if you can list other maps where you are seeing stepping.

    There is not currently a way to control map generation bit-depth. It's all done using 32 bit pixels for the highest accuracy then down-sampled for the selected export format (if needed).

    I do like the idea of having an option to scale vectors in the 0-1 range when exporting to float point formats. I've added it to the list.

    I'm looking forward to your reply, maybe we can get a fix in place before the next update.

    All the best,

  • edited January 2020

    Hi Neil,

    Sorry for long delay.

    Stepping appears on both 16 and 32 bit exr's - values are the same. I tried reproduce this issue in fusion simply converting 32bit baked from mesh displacement in 16bit float file and yes stepping are identical to SM's. So my request is to add true 32 bit support for displacement output for exr's without down-sampling.

    What else bothers me is a some kind of segmentation (?) artifacts on displacement map. I am talking about vertical and horizontal lines across all disp map which becomes like large waves on 999 passes. They are barely noticeable on 2k maps, but 8k is another story and result is completely unusable. Is it possible to fix that? Your algorithm for generation displacement from normals is very good and much much better then anything on the market with simple kernels approximation, but for now it's completely unusable for me as long as that "waves" are there.

    SM has huge potential and I really look forward to when I can use it in our pipeline.


  • edited January 2020

    Also here's bug I found :) When saving displacement to exr SM applies .454* gamma correction assuming that map is in sRGB, so for example - 0.5 mid-value becomes 0.21764 in saved file. It's not a big deal for me because i can apply 2.2 gamma while remapping to -1 to 1 values in fusion anyway but at first I was confused by this.

Sign In or Register to comment.