JSON轉換時去除小數字尾零
2 | 21,695 |
某個ASP.NET MVC Action需要頻繁傳回大型數字陣列,數字大部分是整數,但部分帶有1-2位小數,故陣列採double[]或decimal[]。經Json.NET轉換後有個小問題: 即便是整數,轉換結果也會帶有".0"字尾,例如: double d = 2,Json.NET轉成"2.0",而decimal有個有趣特性,小數尾端的零會被原原本本保存,例如: decimal d = 1.200M,d.ToString()為"1.200",JSON結果也是"1.200"。
本來不是什麼大不了的事,但是當陣列元素一多,原本個位數字1被轉成"1.0",傳輸內容便多兩個Byte,乘上陣列元素個數,佔用頻寬也算可觀。即便IIS有GZip壓縮,但網站效能調校就是這些細節優化所累積出來的。
由於JavaScript不像C#採用強型別,JSON傳回"1"或"1.0"不影響處理結果,我想在Json.NET轉換過程動手腳,當decimal或double是整數時,去除JSON".0"字尾降低傳輸量;若deicmal產出"1.200"這種帶小數零字尾字串,也去除字尾零,輸出"1.2"就好。
準備一個測試ASP.NET MVC Action如下,傳回用亂數產生的1萬筆double陣列,約75%為整數,25%為1位小數: (關於JsonNetResult類別請參考舊文)
static double[] numArray = null;
static double[] GetNumArray()
{
if (numArray == null)
{
Random rnd = new Random(32767);
List<double> lst = new List<double>();
for (int i = 0; i < 10000; i++)
{
var n = rnd.NextDouble() * 10;
if (rnd.Next() % 4 != 0) n = Math.Floor(n);
else n = Math.Round(n, 1);
lst.Add(n);
}
numArray = lst.ToArray();
}
return numArray;
}
public ActionResult LotOfData()
{
return new JsonNetResult()
{
Data = GetNumArray()
};
}
執行結果可以看到5.0, 0.0, 8.0...,一大堆帶有".0"的整數:
Json.NET提供了很棒的擴充性,可自訂JsonConverter針對特殊型別執行指定序列化邏輯。於是我寫了一顆MinifiedNumArrayConveter,繼承JsonConverter,實做CanConvert(),遇到型別為double[]或decimal[]時傳回true,代表支援這兩種型別的轉換;之後Json.NET在遇到double[]或decimal[]時就會呼叫MinifiedNumArrayConveter.WriteJson()。要去除字尾零,我的做法是將double、decimal用.ToString()轉成字串,若出現".0"字尾表示為整數直接去除".0";若字串包含小數點時則用TrimEnd()去掉字尾"0";其餘狀況則直接輸出ToString()。為求效率,我直接呼叫JsonWriter.WriteRawValue輸出處理好的字串,省去讓Json.NET再做一次數字轉字串的程序。另外,去尾零修正只適用WirteJson(),CanRead()一律傳回false代表不處理JSON讀取,並補上一個空的ReadJson()。
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace Web.Models
{
public class MinifiedNumArrayConverter : JsonConverter
{
private void dumpNumArray<T>(JsonWriter writer, T[] array)
{
foreach (T n in array)
{
var s = n.ToString();
//此處可考慮改用string.format("{0:#0.####}")[小數後方#數目依最大小數位數決定]
//感謝網友vencin提供建議
if (s.EndsWith(".0"))
writer.WriteRawValue(s.Substring(0, s.Length - 2));
else if (s.Contains("."))
writer.WriteRawValue(s.TrimEnd('0').TrimEnd('.'));
else
writer.WriteRawValue(s);
}
}
public override void WriteJson(JsonWriter writer, object value,
JsonSerializer serializer)
{
writer.WriteStartArray();
Type t = value.GetType();
if (t == dblArrayType)
dumpNumArray<double>(writer, (double[])value);
else if (t == decArrayType)
dumpNumArray<decimal>(writer, (decimal[])value);
else
throw new NotImplementedException();
writer.WriteEndArray();
}
private Type dblArrayType = typeof(double[]);
private Type decArrayType = typeof(decimal[]);
public override bool CanConvert(Type objectType)
{
if (objectType == dblArrayType || objectType == decArrayType)
return true;
return false;
}
public override bool CanRead
{
get { return false; }
}
public override object ReadJson(JsonReader reader, Type objectType,
object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
}
使用時很簡單,JsonConvert.SerializeObject()時要多傳SerializerSettings參數,用SerializerSettings.Converters.Add()掛上MinifiedNumArrayConverter物件即可。
public ActionResult LotOfFixedData()
{
var res = new JsonNetResult()
{
Data = GetNumArray()
};
res.SerializerSettings.Converters.Add(new MinifiedNumArrayConverter());
return res;
}
如此,JSON裡囉嗦的".0"通通消失了!
實際比較,測試樣本(1萬個數字的陣列,1/4為1位小數,3/4為整數)經MinifiedNumArrayConverter處理,Response大小由40352降到24880,減少38%!
這個技術如果要應用在WebAPI上,要將MinifiedNumArrayConverter加進GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings.Converters,App_Start/WebApiConfig.cs可修改如下:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.Http;
using Web.Models;
namespace Web
{
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
// Web API configuration and services
// Web API routes
config.MapHttpAttributeRoutes();
config.Routes.MapHttpRoute(
name: "DefaultApi",
routeTemplate: "api/{controller}/{id}",
defaults: new { id = RouteParameter.Optional }
);
//強制GET時也傳回JSON,不要傳回XML
GlobalConfiguration.Configuration.Formatters
.XmlFormatter.SupportedMediaTypes.Clear();
//加入自訂序列化轉換邏輯
GlobalConfiguration.Configuration.Formatters
.JsonFormatter.SerializerSettings.Converters.Add(
new MinifiedNumArrayConverter());
}
}
}
如此,所有傳回decimal[]或double[]的WebAPI方法,都會套用MinifiedNumArrayConverter,達到省略小數字尾零的效果。
最後,還有很重要的一點: 加入自訂序列化邏輯是否會嚴重損耗效能呢? 我做了以下實測,比較加入MinifiedNumArrayConverter前後的差異。
public ActionResult Test()
{
StringBuilder sb = new StringBuilder();
sb.AppendLine("<pre>");
var array = GetNumArray();
int times = 200;
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.Converters.Add(new MinifiedNumArrayConverter());
Stopwatch sw = new Stopwatch();
for (int run = 0; run < 5; run++)
{
string res = null;
sw.Reset();
sw.Start();
for (int i = 0; i < times; i++)
{
res = JsonConvert.SerializeObject(array);
}
sw.Stop();
sb.AppendFormat("\nStd JSON: {0:n0}ms \n {1}",
sw.ElapsedMilliseconds, res.Substring(0, 64));
sw.Reset();
sw.Start();
for (int i = 0; i < times; i++)
{
res = JsonConvert.SerializeObject(array, settings);
}
sw.Stop();
sb.AppendFormat("\nMinified JSON: {0:n0}ms \n {1}",
sw.ElapsedMilliseconds, res.Substring(0, 64));
}
string test = "[1.0,2.5,3.0]";
double[] test1 = JsonConvert.DeserializeObject<double[]>(test, settings);
decimal[] test2 = JsonConvert.DeserializeObject<decimal[]>(test, settings);
sb.AppendFormat("\n Deserialization Test: double[] {0}, decimal[] {1}",
test == JsonConvert.SerializeObject(test1) ? "PASS" : "FAIL",
test == JsonConvert.SerializeObject(test2) ? "PASS" : "FAIL"
);
sb.AppendLine("</pre>");
return Content(sb.ToString());
}
public ActionResult Test()
{
StringBuilder sb = new StringBuilder();
sb.AppendLine("<pre>");
var array = GetNumArray();
int times = 200;
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.Converters.Add(new MinifiedNumArrayConverter());
Stopwatch sw = new Stopwatch();
for (int run = 0; run < 5; run++)
{
string res = null;
sw.Reset();
sw.Start();
for (int i = 0; i < times; i++)
{
res = JsonConvert.SerializeObject(array);
}
sw.Stop();
sb.AppendFormat("\nStd JSON: {0:n0}ms \n {1}",
sw.ElapsedMilliseconds, res.Substring(0, 64));
sw.Reset();
sw.Start();
for (int i = 0; i < times; i++)
{
res = JsonConvert.SerializeObject(array, settings);
}
sw.Stop();
sb.AppendFormat("\nMinified JSON: {0:n0}ms \n {1}",
sw.ElapsedMilliseconds, res.Substring(0, 64));
}
string test = "[1.0,2.5,3.0]";
double[] test1 = JsonConvert.DeserializeObject<double[]>(test, settings);
decimal[] test2 = JsonConvert.DeserializeObject<decimal[]>(test, settings);
sb.AppendFormat("\n Deserialization Test: double[] {0}, decimal[] {1}",
test == JsonConvert.SerializeObject(test1) ? "PASS" : "FAIL",
test == JsonConvert.SerializeObject(test2) ? "PASS" : "FAIL"
);
sb.AppendLine("</pre>");
return Content(sb.ToString());
}
程式共跑5次,每次各執行200次1萬個double數字的陣列JSON轉換,比較套用MinifiedNumArrayConverter與否的執行時間,最後順便測試DesrializeObject()在套用MinifiedNumArrayConverter後是否正常。測試結果如下:
Std JSON: 2,385ms [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0 Minified JSON: 1,974ms [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3 Std JSON: 1,615ms [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0 Minified JSON: 1,720ms [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3 Std JSON: 1,316ms [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0 Minified JSON: 2,107ms [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3 Std JSON: 1,767ms [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0 Minified JSON: 1,989ms [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3 Std JSON: 1,591ms [0.4,5.0,0.0,3.8,8.0,7.0,3.0,9.0,7.7,6.0,5.0,7.8,0.5,8.0,2.0,0.0 Minified JSON: 1,786ms [0.4,5,0,3.8,8,7,3,9,7.7,6,5,7.8,0.5,8,2,0,6.9,1,5.5,7,5,6.5,8,3 Deserialization Test: double[] PASS, decimal[] PASS
第一次執行較耗時,依效能測試慣例略過不計,共取四次結果:
1,615ms vs 1,720ms
1,316ms vs 2,107ms
1,767ms vs 1,989ms
1,591ms vs 1,786ms
加入MinifiedNumArrayConverter後速度較慢,但執行兩百萬次的差異約在0.1到0.8秒之間,相較其所節省資料量,評估為划算的投資。
Comments
# by vencin
不能用string.format("{0:#0.#}") 之類的嗎?
# by Jeffrey
to vencin, 試了string.format("{0:#0.####}")很可行(小數後方的#數目依最大容許的小數位數決定),謝謝你的好建議!