XML_SPLIT



XML_SPLIT

NAME
DESCRIPTION
OPTIONS
EXAMPLES
SEE ALSO
TODO
AUTHOR
LICENSE

NAME

  xml_split − cut a big XML file into smaller chunks

DESCRIPTION

"xml_split" takes a (presumably big) XML file and split it in several smaller files. The memory used is the memory needed for the biggest chunk (ie memory is reused for each new chunk).

It can split at a given level in the tree (the default, splits children of the root), or on a condition (using the subset of XPath understood by XML::Twig, so "section" or "/doc/section").

Each generated file is replaced by a processing instruction that will allow "xml_merge" to rebuild the original document. The processing instruction format is "<?merge subdocs=[01] :<filename> ?>"

File names are <file>−<nb>.xml, with <file>−00.xml holding the main document.

OPTIONS

−l <level>

level to cut at: 1 generates a file for each child of the root, 2 for each grand child

defaults to 1

−c <condition>

generate a file for each element that passes the condition

xml_split −c <section> will put each "section" element in its own file (nested sections are handled too)

Note that at the moment this option is a lot slower than using "−l"

−s <size>

generates files of (approximately) <size>. The content of each file is enclosed in a new element ("xml_split::root"), so it’s well-formed XML. The size can be given in bytes, Kb, Mb or Gb.

−g <nb>

groups <nb> elements in a single file. The content of each file is enclosed in a new element ("xml_split::root"), so it’s well-formed XML.

−b <name>

base name for the output, files will be named <base>−<nb><.ext>

<nb> is a sequence number, see below "−−nb_digits" <ext> is an extension, see below "−−extension"

defaults to the original file name (if available) or "out" (if input comes from the standard input)

−n <nb>

number of digits in the sequence number for each file

if more digits than <nb> are needed, then they are used: if "−−nb_digits 2" is used and 112 files are generated they will be named "<file>−01.xml" to "<file>−112.xml"

defaults to 2

−e <ext>

extension to use for generated files

defaults to the original file extension or ".xml"

−i

use XInclude elements instead of Processing Instructions to mark where sub files need to be included

−v

verbose output

Note that this option can slow down processing considerably (by an order of magnitude) when generating lots of small documents

−V

outputs version and exit

−h

short help

−m

man (requires pod2text to be in the path)

EXAMPLES

  xml_split foo.xml             # split at level 1
  xml_split −l 2 foo.xml        # split at level 2
  xml_split −c section foo.xml  # a file is generated for each section element
                                # nested sections are split properly

SEE ALSO

XML::Twig, xml_merge

TODO

optimize the code

any idea welcome! I have already implemented most of what I thought would improve performances.

provide other methods that PIs to keep merge information

XInclude is a good candidate (alpha support added in 0.04).

using entities, which would seem the natural way to do it, doesn’t work, as they make it impossible to have both the main document and the sub docs to be well-formed if the sub docs include sub-sub docs (you can’t have entity declarations in an entity)

AUTHOR

Michel Rodriguez <mirod@cpan.org>

LICENSE

This tool is free software; you can redistribute it and/or modify it under the same terms as Perl itself.







Opportunity


Personal Opportunity - Free software gives you access to billions of dollars of software at no cost. Use this software for your business, personal use or to develop a profitable skill. Access to source code provides access to a level of capabilities/information that companies protect though copyrights. Open source is a core component of the Internet and it is available to you. Leverage the billions of dollars in resources and capabilities to build a career, establish a business or change the world. The potential is endless for those who understand the opportunity.

Business Opportunity - Goldman Sachs, IBM and countless large corporations are leveraging open source to reduce costs, develop products and increase their bottom lines. Learn what these companies know about open source and how open source can give you the advantage.





Free Software


Free Software provides computer programs and capabilities at no cost but more importantly, it provides the freedom to run, edit, contribute to, and share the software. The importance of free software is a matter of access, not price. Software at no cost is a benefit but ownership rights to the software and source code is far more significant.


Free Office Software - The Libre Office suite provides top desktop productivity tools for free. This includes, a word processor, spreadsheet, presentation engine, drawing and flowcharting, database and math applications. Libre Office is available for Linux or Windows.





Free Books


The Free Books Library is a collection of thousands of the most popular public domain books in an online readable format. The collection includes great classical literature and more recent works where the U.S. copyright has expired. These books are yours to read and use without restrictions.


Source Code - Want to change a program or know how it works? Open Source provides the source code for its programs so that anyone can use, modify or learn how to write those programs themselves. Visit the GNU source code repositories to download the source.





Education


Study at Harvard, Stanford or MIT - Open edX provides free online courses from Harvard, MIT, Columbia, UC Berkeley and other top Universities. Hundreds of courses for almost all major subjects and course levels. Open edx also offers some paid courses and selected certifications.


Linux Manual Pages - A man or manual page is a form of software documentation found on Linux/Unix operating systems. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts.