有没有办法在将大型CSV文件转换为JSON时解决Java堆内存错误?

huangapple 未分类评论52阅读模式
英文:

Is there any way I can resolve java heap memory error when converting a large csv file to JSON?

问题

import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;

import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;

public class JacksonProgram {

    public static void main(String[] args) throws Exception {
        File input = new File("C:\\Users\\shivamurthym\\Downloads\\FL_insurance_sample\\FL_insurance_sample.csv");
        File output = new File("C:\\Users\\shivamurthym\\data.json");

        List<Map<?, ?>> data = readObjectsFromCsv(input);
        writeAsJson(data, output);
    }

    public static List<Map<?, ?>> readObjectsFromCsv(File file) throws IOException {
        CsvSchema bootstrap = CsvSchema.emptySchema().withHeader();
        CsvMapper csvMapper = new CsvMapper();
        MappingIterator<Map<?, ?>> mappingIterator = csvMapper.reader(Map.class).with(bootstrap).readValues(file);

        return mappingIterator.readAll();
    }

    public static void writeAsJson(List<Map<?, ?>> data, File file) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.writeValue(file, data);
    }
}
英文:

I'm writing the below to convert CSV file to JSON but getting java heap memory error.

Can anyone help me to write using the Jackson stream API?

import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;

import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;

public class JacksonProgram {

    public static void main(String[] args) throws Exception {
        File input = new File(&quot;C:\\Users\\shivamurthym\\Downloads\\FL_insurance_sample\\FL_insurance_sample.csv&quot;);
        File output = new File(&quot;C:\\Users\\shivamurthym\\data.json&quot;);

        List&lt;Map&lt;?, ?&gt;&gt; data = readObjectsFromCsv(input);
        writeAsJson(data, output);
    }

    public static List&lt;Map&lt;?, ?&gt;&gt; readObjectsFromCsv(File file) throws IOException {
        CsvSchema bootstrap = CsvSchema.emptySchema().withHeader();
        CsvMapper csvMapper = new CsvMapper();
        MappingIterator&lt;Map&lt;?, ?&gt;&gt; mappingIterator = csvMapper.reader(Map.class).with(bootstrap).readValues(file);

        return mappingIterator.readAll();
    }

    public static void writeAsJson(List&lt;Map&lt;?, ?&gt;&gt; data, File file) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.writeValue(file, data);
    }
}

huangapple
  • 本文由 发表于 2020年4月9日 20:01:28
  • 转载请务必保留本文链接:https://java.coder-hub.com/61120699.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定