某個ASP.NET MVC Action需要頻繁傳回大型數字陣列,數字大部分是整數,但部分帶有1-2位小數,故陣列採double[]或decimal[]。經Json.NET轉換後有個小問題: 即便是整數,轉換結果也會帶有".0"字尾,例如: double d = 2,Json.NET轉成"2.0",而decimal有個有趣特性,小數尾端的零會被原原本本保存,例如: decimal d = 1.200M,d.ToString()為"1.200",JSON結果也是"1.200"。

本來不是什麼大不了的事,但是當陣列元素一多,原本個位數字1被轉成"1.0",傳輸內容便多兩個Byte,乘上陣列元素個數,佔用頻寬也算可觀。即便IIS有GZip壓縮,但網站效能調校就是這些細節優化所累積出來的。

由於JavaScript不像C#採用強型別,JSON傳回"1"或"1.0"不影響處理結果,我想在Json.NET轉換過程動手腳,當decimal或double是整數時,去除JSON".0"字尾降低傳輸量;若deicmal產出"1.200"這種帶小數零字尾字串,也去除字尾零,輸出"1.2"就好。

準備一個測試ASP.NET MVC Action如下,傳回用亂數產生的1萬筆double陣列,約75%為整數,25%為1位小數: (關於JsonNetResult類別請參考舊文)

        static double[] numArray = null;
        static double[] GetNumArray()
        {
            if (numArray == null)
            {
                Random rnd = new Random(32767);
                List<double> lst = new List<double>();
                for (int i = 0; i < 10000; i++)
                {
                    var n = rnd.NextDouble() * 10;
                    if (rnd.Next() % 4 != 0) n = Math.Floor(n);
                    else n = Math.Round(n, 1);
                    lst.Add(n);
                }
                numArray = lst.ToArray();
            }
            return numArray;
        }
 
        public ActionResult LotOfData()
        {
            return new JsonNetResult()
            {
                Data = GetNumArray()
            };
        }

執行結果可以看到5.0, 0.0, 8.0...,一大堆帶有".0"的整數:

Json.NET提供了很棒的擴充性,可自訂JsonConverter針對特殊型別執行指定序列化邏輯。於是我寫了一顆MinifiedNumArrayConveter,繼承JsonConverter,實做CanConvert(),遇到型別為double[]或decimal[]時傳回true,代表支援這兩種型別的轉換;之後Json.NET在遇到double[]或decimal[]時就會呼叫MinifiedNumArrayConveter.WriteJson()。要去除字尾零,我的做法是將double、decimal用.ToString()轉成字串,若出現".0"字尾表示為整數直接去除".0";若字串包含小數點時則用TrimEnd()去掉字尾"0";其餘狀況則直接輸出ToString()。為求效率,我直接呼叫JsonWriter.WriteRawValue輸出處理好的字串,省去讓Json.NET再做一次數字轉字串的程序。另外,去尾零修正只適用WirteJson(),CanRead()一律傳回false代表不處理JSON讀取,並補上一個空的ReadJson()。

using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
 
namespace Web.Models
{
    public class MinifiedNumArrayConverter : JsonConverter
    {
 
        private void dumpNumArray<T>(JsonWriter writer, T[] array)
        {
            foreach (T n in array)
            {
                var s = n.ToString();
//此處可考慮改用string.format("{0:#0.####}")[小數後方#數目依最大小數位數決定]
//感謝網友vencin提供建議
                if (s.EndsWith(".0"))
                    writer.WriteRawValue(s.Substring(0, s.Length - 2));
                else if (s.Contains("."))
                    writer.WriteRawValue(s.TrimEnd('0').TrimEnd('.'));
                else
                    writer.WriteRawValue(s);
            }
        }
 
        public override void WriteJson(JsonWriter writer, object value, 
            JsonSerializer serializer)
        {
            writer.WriteStartArray();
            Type t = value.GetType();
            if (t == dblArrayType)
                dumpNumArray<double>(writer, (double[])value);
            else if (t == decArrayType)
                dumpNumArray<decimal>(writer, (decimal[])value);
            else
                throw new NotImplementedException();
            writer.WriteEndArray();
        }
 
        private Type dblArrayType = typeof(double[]);
        private Type decArrayType = typeof(decimal[]);
 
        public override bool CanConvert(Type objectType)
        {
            if (objectType == dblArrayType || objectType == decArrayType)
                return true;
            return false;
        }
 
        public override bool CanRead
        {
            get { return false; }
        }
 
        public override object ReadJson(JsonReader reader, Type objectType, 
            object existingValue, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
 
    }
}

使用時很簡單,JsonConvert.SerializeObject()時要多傳SerializerSettings參數,用SerializerSettings.Converters.Add()掛上MinifiedNumArrayConverter物件即可。

        public ActionResult LotOfFixedData()
        {
            var res = new JsonNetResult()
            {
                Data = GetNumArray()
            };
            res.SerializerSettings.Converters.Add(new MinifiedNumArrayConverter());
            return res;
        }

如此,JSON裡囉嗦的".0"通通消失了!

實際比較,測試樣本(1萬個數字的陣列,1/4為1位小數,3/4為整數)經MinifiedNumArrayConverter處理,Response大小由40352降到24880,減少38%!

這個技術如果要應用在WebAPI上,要將MinifiedNumArrayConverter加進GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings.Converters,App_Start/WebApiConfig.cs可修改如下:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.Http;
using Web.Models;
 
namespace Web
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            // Web API configuration and services
 
            // Web API routes
            config.MapHttpAttributeRoutes();
 
            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
            //強制GET時也傳回JSON,不要傳回XML
            GlobalConfiguration.Configuration.Formatters
                .XmlFormatter.SupportedMediaTypes.Clear();
            //加入自訂序列化轉換邏輯
            GlobalConfiguration.Configuration.Formatters
                .JsonFormatter.SerializerSettings.Converters.Add(
                new MinifiedNumArrayConverter());
        }
    }
}

如此,所有傳回decimal[]或double[]的WebAPI方法,都會套用MinifiedNumArrayConverter,達到省略小數字尾零的效果。

最後,還有很重要的一點: 加入自訂序列化邏輯是否會嚴重損耗效能呢? 我做了以下實測,比較加入MinifiedNumArrayConverter前後的差異。

        public ActionResult Test()
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendLine("<pre>");
            var array = GetNumArray();
            int times = 200;
            JsonSerializerSettings settings = new JsonSerializerSettings();
            settings.Converters.Add(new MinifiedNumArrayConverter());
            Stopwatch sw = new Stopwatch();
 
            for (int run = 0; run < 5; run++)
            {
                string res = null;
                sw.Reset();
                sw.Start();
                for (int i = 0; i < times; i++)
                {
                    res = JsonConvert.SerializeObject(array);
                }
                sw.Stop();
                sb.AppendFormat("\nStd JSON: {0:n0}ms \n {1}",
                    sw.ElapsedMilliseconds, res.Substring(0, 64));
 
                sw.Reset();
                sw.Start();
                for (int i = 0; i < times; i++)
                {
                    res = JsonConvert.SerializeObject(array, settings);
                }
                sw.Stop();
                sb.AppendFormat("\nMinified JSON: {0:n0}ms \n {1}",
                    sw.ElapsedMilliseconds, res.Substring(0, 64));
            }
            string test = "[1.0,2.5,3.0]";
            double[] test1 = JsonConvert.DeserializeObject<double[]>(test, settings);
            decimal[] test2 = JsonConvert.DeserializeObject<decimal[]>(test, settings);
            sb.AppendFormat("\n Deserialization Test: double[] {0}, decimal[] {1}",
                test == JsonConvert.SerializeObject(test1) ? "PASS" : "FAIL",
                test == JsonConvert.SerializeObject(test2) ? "PASS" : "FAIL"
                );
            sb.AppendLine("</pre>");
            return Content(sb.ToString());
        }
        public ActionResult Test()
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendLine("<pre>");
            var array = GetNumArray();
            int times = 200;
            JsonSerializerSettings settings = new JsonSerializerSettings();
            settings.Converters.Add(new MinifiedNumArrayConverter());
            Stopwatch sw = new Stopwatch();
 
            for (int run = 0; run < 5; run++)
            {
                string res = null;
                sw.Reset();
                sw.Start();
                for (int i = 0; i < times; i++)
                {
                    res = JsonConvert.SerializeObject(array);
                }
                sw.Stop();
                sb.AppendFormat("\nStd JSON: {0:n0}ms \n {1}",
                    sw.ElapsedMilliseconds, res.Substring(0, 64));
 
                sw.Reset();
                sw.Start();
                for (int i = 0; i < times; i++)
                {
                    res = JsonConvert.SerializeObject(array, settings);
                }
                sw.Stop();
                sb.AppendFormat("\nMinified JSON: {0:n0}ms \n {1}",
                    sw.ElapsedMilliseconds, res.Substring(0, 64));
            }
            string test = "[1.0,2.5,3.0]";
            double[] test1 = JsonConvert.DeserializeObject<double[]>(test, settings);
            decimal[] test2 = JsonConvert.DeserializeObject<decimal[]>(test, settings);
            sb.AppendFormat("\n Deserialization Test: double[] {0}, decimal[] {1}",
                test == JsonConvert.SerializeObject(test1) ? "PASS" : "FAIL",
                test == JsonConvert.SerializeObject(test2) ? "PASS" : "FAIL"
                );
            sb.AppendLine("</pre>");
            return Content(sb.ToString());
        }

程式共跑5次,每次各執行200次1萬個double數字的陣列JSON轉換,比較套用MinifiedNumArrayConverter與否的執行時間,最後順便測試DesrializeObject()在套用MinifiedNumArrayConverter後是否正常。測試結果如下:

Std JSON: 2,385ms 
 [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0
Minified JSON: 1,974ms 
 [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3
Std JSON: 1,615ms 
 [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0
Minified JSON: 1,720ms 
 [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3
Std JSON: 1,316ms 
 [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0
Minified JSON: 2,107ms 
 [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3
Std JSON: 1,767ms 
 [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0
Minified JSON: 1,989ms 
 [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3
Std JSON: 1,591ms 
 [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0
Minified JSON: 1,786ms 
 [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3
 Deserialization Test: double[] PASS, decimal[] PASS

第一次執行較耗時,依效能測試慣例略過不計,共取四次結果:

1,615ms vs 1,720ms
1,316ms vs 2,107ms
1,767ms vs 1,989ms
1,591ms vs 1,786ms

加入MinifiedNumArrayConverter後速度較慢,但執行兩百萬次的差異約在0.1到0.8秒之間,相較其所節省資料量,評估為划算的投資。


Comments

# by vencin

不能用string.format("{0:#0.#}") 之類的嗎?

# by Jeffrey

to vencin, 試了string.format("{0:#0.####}")很可行(小數後方的#數目依最大容許的小數位數決定),謝謝你的好建議!

Post a comment