Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OutOfMemoryError if xml file big #1

Open
GoogleCodeExporter opened this issue Jan 27, 2016 · 5 comments
Open

OutOfMemoryError if xml file big #1

GoogleCodeExporter opened this issue Jan 27, 2016 · 5 comments

Comments

@GoogleCodeExporter
Copy link

What steps will reproduce the problem?
1. get bix xml file ( I use 0.5Gb )
2. run tool
3. receive error

What is the expected output? What do you see instead?
proper cvs file

What version of the product are you using? On what operating system?
0.0.1
WinXP

Please provide any additional information below.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Unknown Source)
        at java.lang.String.<init>(Unknown Source)
        at com.sun.org.apache.xerces.internal.xni.XMLString.toString(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.characters(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown Source)
        at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown Source)
        at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
        at xml2csvconv.Main.convert(Main.java:194)
        at xml2csvconv.Main.main(Main.java:493)

Original issue reported on code.google.com by wot...@gmail.com on 28 Mar 2011 at 6:54

@GoogleCodeExporter
Copy link
Author

you can try to increase heap space by adding e.g. "-Xms1024m" to the .bat-file. 
COr use command line "java -Xms1024m -jar xml2csv-conv.jar %*", if 1 GB RAM is 
enough for your file. I just try it with a > 200 MB XML-file and it seems to 
work, but takes quite a lot time. Smaller files did work!

Original comment by christop...@gmail.com on 27 Mar 2014 at 3:35

@GoogleCodeExporter
Copy link
Author

I have a XML of 2GB and I tried everything with the memory settings and I still 
get OutOfMemoryExc. 

Im running on Win8 & 6GB RAM.

Original comment by seskici2...@gmail.com on 6 Aug 2014 at 7:08

@GoogleCodeExporter
Copy link
Author

I have tried using 10 GB RAM on 5 GB of XML file...Still i am facing the same 
problem..I am running on 64GB RAM linux machine...Can any body suggest

Original comment by kumar.pa...@gmail.com on 25 Sep 2014 at 10:17

@GoogleCodeExporter
Copy link
Author

Hi, I am using this jar file for conversion of XML. But i am not getting in 
proper CSV format after converting
I used following command 
java -jar xml2csv-conv.jar -i entry,feature,reference,sequence,citation,person  
uniprotsample.xml demo.csv

Here i am attaching sample xml file and output of it (CSV file)
Please find the attachments. My original file size is 5 GB

Can any body suggest
Pavan Kumar Alluri

Original comment by kumar.pa...@gmail.com on 3 Dec 2014 at 7:07

Attachments:

@GoogleCodeExporter
Copy link
Author

I had the same issue with a file that was just over 500MB. Increased the memory 
of the JVM and left it to run over night. While the error wasn't produced, the 
process hadn't completed in over 12 hours. Killed the process in the end as the 
computer was very slow in responding.

Original comment by jshpi...@gmail.com on 30 May 2015 at 5:05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant