英文:
FasterXML performance regression when dealing with list?
问题
我正在评估 fasterXML(版本 2.11.0)在不同配置("simple" objectMapper、使用 afterburner、smile 等)下的性能,而在我的 jmh 性能测试中,我发现了一些奇怪的结果。似乎对于反序列化列表的性能非常非常差。以下是用于测试的方法:
protected void testMapper(List<Persona> persone, Blackhole bh, ObjectMapper jsonMapper) throws Exception {
String personeAsStr = jsonMapper.writeValueAsString(persone);
bh.consume(personeAsStr);
List<Persona> personeDeser = jsonMapper.readValue(personeAsStr, List.class);
bh.consume(personeDeser);
}
如果我使用一个包含 persona 属性的对象,而不是 List
protected void testMapperPersone(PersoneForBenchmark personeForBenchmark, Blackhole bh, ObjectMapper jsonMapper) throws Exception {
String personeAsStr = jsonMapper.writeValueAsString(personeForBenchmark);
bh.consume(personeAsStr);
PersoneForBenchmark personeDeser = jsonMapper.readValue(personeAsStr, PersoneForBenchmark.class);
bh.consume(personeDeser);
}
其中 PersonaForBenchmark 类似于:
public class PersoneForBenchmark {
private Persona persona0;
private Persona persona1;
private Persona persona2;
private Persona persona3;
private Persona persona4;
private Persona persona5;
private Persona persona6;
private Persona persona7;
private Persona persona8;
private Persona persona9;
//加上 getter 和 setter
}
性能结果如下:
- FasterXMLPerformance.fasterXMLStandard 19,504 次/秒
- FasterXMLPerformance.fasterXMLStandardPersone 10772,836 次/秒
我尝试过使用 TypeReference,一个持有列表的对象,但结果基本相同。另一个让我困惑的方面是测试的吞吐量(针对列表测试用例)在每次迭代后都会下降。以下是 jmh 日志的示例:
运行进度:18.75% 完成,预计剩余时间 00:16:30
分支:1/1
热身迭代 1:218.673 次/秒
热身迭代 2:83.445 次/秒
热身迭代 3:57.964 次/秒
热身迭代 4:46.002 次/秒
热身迭代 5:39.482 次/秒
热身迭代 6:34.743 次/秒
热身迭代 7:31.322 次/秒
热身迭代 8:28.361 次/秒
热身迭代 9:26.241 次/秒
热身迭代 10:24.520 次/秒
迭代 1:23.291 次/秒
迭代 2:22.062 次/秒
迭代 3:20.236 次/秒
迭代 4:19.886 次/秒
迭代 5:19.379 次/秒
完整的测试套件在此处可用。有人可以告诉我是否做错了什么,或者这确实是性能退化吗?
英文:
I'm evaluating fasterXML (v 2.11.0) performance under different configurations ("simple" objectMapper, using afterburner, smile, etc) and during my jmh performance tests I found some strange results.
It seems that the performance for de-serializing a list is very very poor.
Here is the method used for the test
protected void testMapper(List<Persona> persone, Blackhole bh,ObjectMapper jsonMapper) throws Exception {
String personeAsStr=jsonMapper.writeValueAsString(persone);
bh.consume(personeAsStr);
List<Persona> personeDeser=jsonMapper.readValue(personeAsStr,List.class);
bh.consume(personeDeser);
}
If instead of List<Persona> I use an object containing the persona as properties the performance is totally different.
Here is the code used for the other test
protected void testMapperPersone(PersoneForBenchmark personeForBenchmark, Blackhole bh,ObjectMapper jsonMapper) throws Exception {
String personeAsStr=jsonMapper.writeValueAsString(personeForBenchmark);
bh.consume(personeAsStr);
PersoneForBenchmark personeDeser=jsonMapper.readValue(personeAsStr,PersoneForBenchmark.class);
bh.consume(personeDeser);
}
Where PersonaForBenchmark is something like:
public class PersoneForBenchmark {
private Persona persona0;
private Persona persona1;
private Persona persona2;
private Persona persona3;
private Persona persona4;
private Persona persona5;
private Persona persona6;
private Persona persona7;
private Persona persona8;
private Persona persona9;
//plus getters and setters
}
The performance results are the following one:
FasterXMLPerformance.fasterXMLStandard 19,504 ops/s
FasterXMLPerformance.fasterXMLStandardPersone 10772,836 ops/s
I tried to use a TypeReference, an Object holding the list, but the results are essentially the same. Another aspect that is puzzling me is that the throughput of the test (for the list test case) decrease after each iteration. Here is an example of jmh log:
Run progress: 18,75% complete, ETA 00:16:30
Fork: 1 of 1
Warmup Iteration 1: 218,673 ops/s
Warmup Iteration 2: 83,445 ops/s
Warmup Iteration 3: 57,964 ops/s
Warmup Iteration 4: 46,002 ops/s
Warmup Iteration 5: 39,482 ops/s
Warmup Iteration 6: 34,743 ops/s
Warmup Iteration 7: 31,322 ops/s
Warmup Iteration 8: 28,361 ops/s
Warmup Iteration 9: 26,241 ops/s
Warmup Iteration 10: 24,520 ops/s
Iteration 1: 23,291 ops/s
Iteration 2: 22,062 ops/s
Iteration 3: 20,236 ops/s
Iteration 4: 19,886 ops/s
Iteration 5: 19,379 ops/s
The full test suite is available here.
Can anyone tell me if I'm doing something wrong or is it really a performance regression?
答案1
得分: 0
我看不出测试设置中有明显的问题,我认为你使用了jmh
是一个很好的性能框架,这很棒。
我同意你看到的结果令人困惑,尤其是在测试运行过程中性能下降。
因此,我建议你在jackson-databind
的GitHub存储库上提交一个问题,附上上面提到的一些信息,并链接到这个问题。我将在Gitter聊天(https://gitter.im/FasterXML/jackson-databind)上提供一些更多的建议。
但一个立即有用的事情是了解是否具体的旧版本没有这个问题。在2.11版本中有关于类型解析的更改(随着时间的推移进行了调整),这可能会导致某些用例的回归 - 我们确实对性能进行基准测试,但主要是使用更简单的List
(例如List<String>
)进行测试。
英文:
I can not see an immediate problem with test setup, and I think it is great that you are using jmh
which is a solid good performance framework.
I agree that results you see are puzzling; and especially so degrading performance over test runs.
With that, I would suggest you file an issue for jackson-databind
github repo, with some of information from above, and link to this question.
I will add some more suggestions on Gitter chat (https://gitter.im/FasterXML/jackson-databind)
But one immediately useful thing would be to know if specific older versions did not have this issue. There were changes in 2.11 about type resolution (it has been tweaker over time) which could have caused regression for some use case -- we do benchmark performance but mostly with just simpler List
s (List<String>
etc).
专注分享java语言的经验与见解,让所有开发者获益!
评论