kernowforsaxon-help Mailing List for Kernow

Good morning,

does Kernow support XQuery modules, i.e. can I refer to an XQuery library module by referring to its module namespace in the XQuery header via “import module namespace nsshort=nslong;" ? Itried in the sandbox mode so far and it gave me
Error on line 2 column 1 -   XQST0059 XQuery static error near #... -  Cannot locate module for namespace nslong. I put the module (extension .xquery) into the same directory, Kerno.exe is located in and the module starts with

xquery version "1.0" encoding "UTF-8";
module namespace p="nslong";

Thanks & regards,
Juliane.



Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com

 

 

From: Ed Calhoun [mailto:cc...@sa...] 
Sent: Monday, December 01, 2014 7:14 PM
To: 'ker...@li...'
Subject: Logging output messages with xsl:message

 

Hi Andrew,

 

I'd like to know if there is a way that I can write output messages to
Kernow's message/error dialog.  I tried using xsl:message without any luck
but probably don't have things configured properly.  I'm using SaxonHE 9.5.

 

Kind regards,

Ed

 

Has anyone successfully purchased a license for Kernow? I can’t get a response from the developer.


Marc Feickert
Enterprise Search Architect  |  Enterprise Engineering Services
536 Chapel Hills Dr. Suite 120, Colorado Springs, CO, 80920
+1 719-452-4193
mar...@op...<mailto:mar...@op...>
[Description: Description: optum logo]
               [Small-BDGE-Cert-Know-Mngr-v6-GRY-RGB]


This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.

Hello,

We are trying Kernow 1.8.0.1 ant task for processing directories 
transformations.

We have a licence for Saxon PE 9.6.0.2 which is correctly detected.

Our stylesheet is using a pro feature (the reason we have Saxon PE) to 
chain different stylesheet  :   <xsl:output method="xml" indent="yes" 
*saxon:next-in-chain="rsc2rdf2.xsl"*/>

The problem occurs when we are using kernow-directory ant task 
(*net.sf.kernow.DirectoryTransform* class) :


[devel@tempete ant]$ ant kernow-directory
Buildfile: /home/devel/kernow/ant/build.xml

kernow-directory:
      [java] Compiling stylesheet...
      [java] Exception in thread "main" 
net.sf.saxon.trans.LicenseException: Requested feature (custom 
serialization {http://saxon.sf.net/}next-in-chain) requires Saxon-PE. 
You are using Saxon-PE software, *but the Configuration is an instance 
of net.sf.saxon.Configuration; to use this feature you need to create an 
instance of com.saxonica.config.ProfessionalConfiguration*
      [java]     at 
net.sf.saxon.Configuration.checkLicensedFeature(Configuration.java:518)
      [java]     at 
net.sf.saxon.lib.SaxonOutputKeys.checkExtensions(SaxonOutputKeys.java:395)
      [java]     at 
net.sf.saxon.lib.SaxonOutputKeys.checkOutputProperty(SaxonOutputKeys.java:345)
      [java]     at 
net.sf.saxon.style.XSLOutput.checkAndPut(XSLOutput.java:371)
      [java]     at 
net.sf.saxon.style.XSLOutput.gatherOutputProperties(XSLOutput.java:322)
      [java]     at 
net.sf.saxon.style.StylesheetPackage.gatherOutputProperties(StylesheetPackage.java:986)
      [java]     at 
net.sf.saxon.style.StylesheetPackage.updatePreparedStylesheet(StylesheetPackage.java:1293)
      [java]     at 
net.sf.saxon.style.Compilation.compileSingletonPackage(Compilation.java:101)
      [java]     at 
net.sf.saxon.s9api.XsltCompiler.compile(XsltCompiler.java:543)
      [java]     at 
net.sf.kernow.transform.DirectoryTransformer.runDirectoryTransform(SourceFile:102)
      [java]     at *net.sf.kernow.DirectoryTransform.main(SourceFile:44)*

BUILD FAILED
/home/devel/kernow/ant/build.xml:119: Java returned: 1

Do you have an idea to solve this pb ? Thanks

Thomas

PS : if info needed, we use the task this way :

<target name="kernow-directory">
         <fail message="Property: 'input.dir' not set" unless="input.dir"/>
         <fail message="Property: 'xslt' not set" unless="xslt"/>
         <fail message="Property: 'output.dir' not set" 
unless="output.dir"/>
         <fail message="Property: 'xslt.params' not set" 
unless="xslt.params"/>
         <java classname="net.sf.kernow.DirectoryTransform" fork="true" 
failonerror="true">
             <jvmarg 
value="-Dkernow.config=${kernow.home}/kernow1dot8.config"/>
             <jvmarg value="-Xmx4096m"/>
             <jvmarg value="-Xms1024m"/>
             <classpath>
                 <pathelement location="${kernow.home}"/>
                 <fileset dir="${kernow.home}">
                     <include name="lib/*.jar"/>
                     <include name="kernow.jar"/>
                 </fileset>
             </classpath>
             <arg value="${input.dir}"/>
             <arg value="${xslt}"/>
             <arg value="${output.dir}"/>
             <arg line="${xslt.params}"/>
         </java>
     </target>

Hi Andrew,

 

I would like to use Kernow on a modular input file, something like:

 

<?xml version="1.0" encoding="UTF-8"?>

<library id="OTLXI" xmlns:xi="http://www.w3.org/2001/XInclude">

  <name>RWS Modellenbibliotheek</name>

  <definition>De RWS Modellenbibliotheek is een verzameling van modellen en
hun inter-model relaties.</definition>

   <xi:include href="Export_Generic_Model_otl_doc-2014-05-07.xml"
parse="xml" />

   <xi:include href="Export_Generic_Model_otl-doctype-2014-05-07.xml"
parse="xml" />

   <xi:include href="Export_Configuration_Model-2014-05-08.xml" parse="xml"
/>

</library>

 

However, I don't know how to turn on the XInclude processing. From the Saxon
documentation I retrieved a mention of the -xi Saxon commandline option.
It's not included with the Saxon options I can set in Kernow. 

 

Could you help me?

 

Kind regards,

 

Mick Baggen

IJsselstein Ut

The Netherlands

 

 

 

ok, then I would try to simplify a couple of rows and clean out the 
questionable content. You need to get content you need to test with. 
then once you have everything working, start to introduce the content 
that you think might be causing the problem.

..dan

At 08:06 PM 7/15/2013, you wrote:
>thanks for your replay, Danny
>
>pretty sure about the prolog error since message includes an option 
>(Stylus Studio) to jump to the source of the error, in this case, 
>the CSV file is opened with the cursor sitting at the beginning.
>
>Will take a look at the Crane stylesheet...think I have already 
>downloaded it, just have not tried it yet.
>
>My main concern is the likely variability of cell content.  For 
>example, publishers ONIX product records are encoded by folks with 
>different levels of knowledge of ONIX for books; frequently includes 
>HTML markup in some elements (something that is easy to solve 
>however with the workflow I alluded to).
>
>My objective is to provide an affordable option for libraries that 
>would allow them to use a couple of stylesheets to transform CSV 
>versions of spreadsheets with the library application I mentioned 
>(MarcEdit, created by the inestimable Terry Reece, the Ohio State University).
>
>thanks again,
>dana (danny to my mom)
>
>
>On Mon, Jul 15, 2013 at 8:41 PM, Dan Vint 
><<mailto:dv...@dv...>dv...@dv...> wrote:
>At 06:25 PM 7/15/2013, you wrote:
> >The transformation could not get beyond the initial look at the
> >source file; error message something to the effect, something not
> >allowed in the prolog.
>
>Are you are sure the error was about the file being processed and not
>the stylesheet? Sometimes the files will get extra characters before
>the xml prolog statement. You might want to check that.
>
>As far as valid CSV that just depends upon who wrote the spec for the
>stylesheet as to what they expect to find. The prolog error message
>you mention sounds like an issue with the XSLT not the source file
>you are processing. If you want to test the source file, I would just
>take a couple of rows of the file, and clean them up. So I would look
>at the use of quotes (single and double), MS products like to
>substitute curly quotes and long dashes which can be hard to spot.
>I'm not familiar with this stylesheet, but I'm assuming it is looking
>for tabs as the separation character, so you might want to look for
>missing tabs such that not all the columns are there. Basically I
>would simplify the source as much as possible to create a test source
>file to verify you have everything working.
>
>If you still can't get it to work, Ken Holman has a  general purpose
>CSV processing stylesheet that you might want to try out as well.
><<http://www.CraneSoftwrights.com/resources/#csv>http://www.CraneSoftwrights.com/resources/#csv>http://www.CraneSoftwrights.com/resources/#csv
>
>..dan
>---------------------------------------------------------------------------
>Danny Vint
>
>Panoramic Photography
><http://www.dvint.com>http://www.dvint.com
>
>voice: <tel:619-647-5780>619-647-5780
>
>
>
>------------------------------------------------------------------------------
>See everything from the browser to the database with AppDynamics
>Get end-to-end visibility with application monitoring from AppDynamics
>Isolate bottlenecks and diagnose root cause in seconds.
>Start your free trial of AppDynamics Pro today!
><http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk>http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>_______________________________________________
>Kernowforsaxon-help mailing list
><mailto:Ker...@li...>Ker...@li...
>https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>
>
>
>
>--
>Dana Pearson
><http://dbpearsonmlis.com>dbpearsonmlis.com
>------------------------------------------------------------------------------
>See everything from the browser to the database with AppDynamics
>Get end-to-end visibility with application monitoring from AppDynamics
>Isolate bottlenecks and diagnose root cause in seconds.
>Start your free trial of AppDynamics Pro today!
>http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>_______________________________________________
>Kernowforsaxon-help mailing list
>Ker...@li...
>https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help

---------------------------------------------------------------------------
Danny Vint

Panoramic Photography
http://www.dvint.com

voice: 619-647-5780
     

thanks for your replay, Danny

pretty sure about the prolog error since message includes an option (Stylus
Studio) to jump to the source of the error, in this case, the CSV file is
opened with the cursor sitting at the beginning.

Will take a look at the Crane stylesheet...think I have already downloaded
it, just have not tried it yet.

My main concern is the likely variability of cell content.  For example,
publishers ONIX product records are encoded by folks with different levels
of knowledge of ONIX for books; frequently includes HTML markup in some
elements (something that is easy to solve however with the workflow I
alluded to).

My objective is to provide an affordable option for libraries that would
allow them to use a couple of stylesheets to transform CSV versions of
spreadsheets with the library application I mentioned (MarcEdit, created by
the inestimable Terry Reece, the Ohio State University).

thanks again,
dana (danny to my mom)


On Mon, Jul 15, 2013 at 8:41 PM, Dan Vint <dv...@dv...> wrote:

> At 06:25 PM 7/15/2013, you wrote:
> >The transformation could not get beyond the initial look at the
> >source file; error message something to the effect, something not
> >allowed in the prolog.
>
> Are you are sure the error was about the file being processed and not
> the stylesheet? Sometimes the files will get extra characters before
> the xml prolog statement. You might want to check that.
>
> As far as valid CSV that just depends upon who wrote the spec for the
> stylesheet as to what they expect to find. The prolog error message
> you mention sounds like an issue with the XSLT not the source file
> you are processing. If you want to test the source file, I would just
> take a couple of rows of the file, and clean them up. So I would look
> at the use of quotes (single and double), MS products like to
> substitute curly quotes and long dashes which can be hard to spot.
> I'm not familiar with this stylesheet, but I'm assuming it is looking
> for tabs as the separation character, so you might want to look for
> missing tabs such that not all the columns are there. Basically I
> would simplify the source as much as possible to create a test source
> file to verify you have everything working.
>
> If you still can't get it to work, Ken Holman has a  general purpose
> CSV processing stylesheet that you might want to try out as well.
> <http://www.CraneSoftwrights.com/resources/#csv>
> http://www.CraneSoftwrights.com/resources/#csv
>
> ..dan
> ---------------------------------------------------------------------------
> Danny Vint
>
> Panoramic Photography
> http://www.dvint.com
>
> voice: 619-647-5780
>
>
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>



-- 
Dana Pearson
dbpearsonmlis.com

At 06:25 PM 7/15/2013, you wrote:
>The transformation could not get beyond the initial look at the 
>source file; error message something to the effect, something not 
>allowed in the prolog.

Are you are sure the error was about the file being processed and not 
the stylesheet? Sometimes the files will get extra characters before 
the xml prolog statement. You might want to check that.

As far as valid CSV that just depends upon who wrote the spec for the 
stylesheet as to what they expect to find. The prolog error message 
you mention sounds like an issue with the XSLT not the source file 
you are processing. If you want to test the source file, I would just 
take a couple of rows of the file, and clean them up. So I would look 
at the use of quotes (single and double), MS products like to 
substitute curly quotes and long dashes which can be hard to spot. 
I'm not familiar with this stylesheet, but I'm assuming it is looking 
for tabs as the separation character, so you might want to look for 
missing tabs such that not all the columns are there. Basically I 
would simplify the source as much as possible to create a test source 
file to verify you have everything working.

If you still can't get it to work, Ken Holman has a  general purpose 
CSV processing stylesheet that you might want to try out as well. 
<http://www.CraneSoftwrights.com/resources/#csv>http://www.CraneSoftwrights.com/resources/#csv

..dan
---------------------------------------------------------------------------
Danny Vint

Panoramic Photography
http://www.dvint.com

voice: 619-647-5780
     

I would like to use Andrew Welch's CSV to XML stylesheet for a project
designed for libraries.  Some libraries are receiving metadata for ebooks
in ONIX for bulk purchases.  However, many, perhaps most are supplying the
metadata in spreadsheets.

I use Stylus Studio Enterprise 2010 for most of my work but the Andrew's
stylesheet would not work although I tried adding the standalone attribute
to the output element and supplied the path to the source file in the param
in the stylesheet.

The transformation could not get beyond the initial look at the source
file; error message something to the effect, something not allowed in the
prolog.

So, downloaded Kernow using Java Web Start option.  Pretty excited when I
got an XML output but discovered that the file rearranged output so that
the root element and its close were jumbled within the file rendering it
corrupted beyond reasonable repair.

Looking again at the page,
http://andrewjwelch.com/code/xslt/csv/csv-to-xml_v2.html
I took note of the sentence:

"If this transform fails for a valid CSV file, let me know!"

So, I examined the source CSV file and saw cell content that could be the
source of the problems.  For example, quotations characters of reviews of
an ebook within a cell that also included content not within the quotation
characters.  E.g., quote within quotation characters followed by a
punctuation character (not ascii) and source of quote, eg, publication
name, maybe an author.

Not looking for a solution but better understanding of what valid CSV is.
 Impressed with Kernow, very nice, easy to use application.  It is easy for
me to use a Stylus Studio utility to do the transformation but had hoped
that I could create a work flow for libraries that would not require it.  A
library application freely available to libraries offers a very useful XSLT
tool and includes latest version of Saxon along with some default processor
and .NET options.

In particular, I would like to know if valid CSV includes the notion of
regular cell size or length perhaps.  More generally, what approaches might
one take if CSV source is as untidy as I expect would be received from most
publishers?  Is this truly a dead end for me?

regards,
dana

-- 
Dana Pearson
dbpearsonmlis.com

Hello Michéle,

Instead of this

> <xsl:template match="text()" />


I would add the following to remove all whitespace from the source:

<xsl:strip-space select="*" />

Also, there is a white-space related option in the Saxon Option settings panel. Try this as well.

Best regards,

- Michael Müller-Hillebrand

Hello,

I'm using Kernow 1.7.2 on Mac OS X Snow Leopard.

I use the following xml source:


<?xml version="1.0" ?>
<!-- Data for XPath, XQuery, and XSLT demos -->

<!DOCTYPE Bookstore [
   <!ELEMENT Bookstore (Book | Magazine)*>
   <!ELEMENT Book (Title, Authors, Remark?)>
   <!ATTLIST Book ISBN CDATA #REQUIRED
                  Price CDATA #REQUIRED
                  Edition CDATA #IMPLIED>
   <!ELEMENT Magazine (Title)>
   <!ATTLIST Magazine Month CDATA #REQUIRED Year CDATA #REQUIRED>
   <!ELEMENT Title (#PCDATA)>
   <!ELEMENT Authors (Author+)>
   <!ELEMENT Remark (#PCDATA)>
   <!ELEMENT Author (First_Name, Last_Name)>
   <!ELEMENT First_Name (#PCDATA)>
   <!ELEMENT Last_Name (#PCDATA)>
]>

<Bookstore>
   <Book ISBN="ISBN-0-13-713526-2" Price="85" Edition="3rd">
      <Title>A First Course in Database Systems</Title>
      <Authors>
         <Author>
            <First_Name>Jeffrey</First_Name>
            <Last_Name>Ullman</Last_Name>
         </Author>
         <Author>
            <First_Name>Jennifer</First_Name>
            <Last_Name>Widom</Last_Name>
         </Author>
      </Authors>
   </Book>
   <Book ISBN="ISBN-0-13-815504-6" Price="100">
      <Title>Database Systems: The Complete Book</Title>
      <Authors>
         <Author>
            <First_Name>Hector</First_Name>
            <Last_Name>Garcia-Molina</Last_Name>
         </Author>
         <Author>
            <First_Name>Jeffrey</First_Name>
            <Last_Name>Ullman</Last_Name>
         </Author>
         <Author>
            <First_Name>Jennifer</First_Name>
            <Last_Name>Widom</Last_Name>
         </Author>
      </Authors>
      <Remark>
         Buy this book bundled with "A First Course" - a great deal!
      </Remark>
   </Book>
   <Book ISBN="ISBN-0-11-222222-3" Price="50">
      <Title>Hector and Jeff's Database Hints</Title>
      <Authors>
         <Author>
            <First_Name>Jeffrey</First_Name>
            <Last_Name>Ullman</Last_Name>
         </Author>
         <Author>
            <First_Name>Hector</First_Name>
            <Last_Name>Garcia-Molina</Last_Name>
         </Author>
      </Authors>
      <Remark>An indispensible companion to your textbook</Remark>
   </Book>
   <Book ISBN="ISBN-9-88-777777-6" Price="25">
      <Title>Jennifer's Economical Database Hints</Title>
      <Authors>
         <Author>
            <First_Name>Jennifer</First_Name>
            <Last_Name>Widom</Last_Name>
         </Author>
      </Authors>
   </Book>
   <Magazine Month="January" Year="2009">
       <Title>National Geographic</Title>
   </Magazine>
   <Magazine Month="February" Year="2009">
       <Title>National Geographic</Title>
   </Magazine>
   <Magazine Month="February" Year="2009">
       <Title>Newsweek</Title>
   </Magazine>
   <Magazine Month="March" Year="2009">
       <Title>Hector and Jeff's Database Hints</Title>
   </Magazine>
</Bookstore>


and the following xslt query:

<xsl:stylesheet version="2.0" xmlns:xsl="
http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes" omit-xml-declaration="yes" />

<xsl:template match="Book[@Price &lt; 90]">
   <xsl:copy-of select="." />
</xsl:template>

<xsl:template match="text()" />

</xsl:stylesheet>


The result is as follows:


<Book ISBN="ISBN-0-13-713526-2" Price="85" Edition="3rd">
      <Title>A First Course in Database Systems</Title>
      <Authors>
         <Author>
            <First_Name>Jeffrey</First_Name>
            <Last_Name>Ullman</Last_Name>
         </Author>
         <Author>
            <First_Name>Jennifer</First_Name>
            <Last_Name>Widom</Last_Name>
         </Author>
      </Authors>
   </Book>
<Book ISBN="ISBN-0-11-222222-3" Price="50">
      <Title>Hector and Jeff's Database Hints</Title>
      <Authors>
         <Author>
            <First_Name>Jeffrey</First_Name>
            <Last_Name>Ullman</Last_Name>
         </Author>
         <Author>
            <First_Name>Hector</First_Name>
            <Last_Name>Garcia-Molina</Last_Name>
         </Author>
      </Authors>
      <Remark>An indispensible companion to your textbook</Remark>
   </Book>
<Book ISBN="ISBN-9-88-777777-6" Price="25">
      <Title>Jennifer's Economical Database Hints</Title>
      <Authors>
         <Author>
            <First_Name>Jennifer</First_Name>
            <Last_Name>Widom</Last_Name>
         </Author>
      </Authors>
   </Book>

Hope you see that the closing </Book> is indented with three spaces
contrary to the opening <Book> with 0 space.

When I run under the query from the command line under the decompressed
saxon9he.jar with:

java net.sf.saxon.Transform -s:"BookstoreQ.xml" -xsl:"Query.xml"
-o:"Result.xml"

(BookstoreQ.xml being the xml above source and Query.xml being the xslt
query above) I got correct indentation:

<Book ISBN="ISBN-0-13-713526-2" Price="85" Edition="3rd">
   <Title>A First Course in Database Systems</Title>
   <Authors>
      <Author>
         <First_Name>Jeffrey</First_Name>
         <Last_Name>Ullman</Last_Name>
      </Author>
      <Author>
         <First_Name>Jennifer</First_Name>
         <Last_Name>Widom</Last_Name>
      </Author>
   </Authors>
</Book>
<Book>
...
</Book>

What could be the problem? Is there a way to solve it inside Kernow?

Thanks in advance for any help.


-- 
Regards,
Michèle Garoche

Hi,

You can try replacing the Saxon jar in the kernow/lib directory with
the newer version, however because Kernow uses some internal apis in
Saxon there's no guarantee that will work.

There is a new version of Kernow in progress, but I will probably wait
for Saxon 9.5 before releasing it.

cheers
andrew

On 27 June 2012 09:30, Ben Stover <bxs...@ya...> wrote:
> If I open the "About" box of Kernow v1.7 then there is Saxon HE 9.3.0.2 mentioned as reference implementation of Kernow.
>
> How do I tell Kernow to use (the newer) Saxon HE v9.4.0.4 instead and where this location is installed?
>
> Will there be any Kernow updates in the future?
> Or is development of Kernow closed?
>
> Thank you
> Ben
>
>



-- 
Andrew Welch
http://andrewjwelch.com

If I open the "About" box of Kernow v1.7 then there is Saxon HE 9.3.0.2 mentioned as reference implementation of Kernow.

How do I tell Kernow to use (the newer) Saxon HE v9.4.0.4 instead and where this location is installed?

Will there be any Kernow updates in the future?
Or is development of Kernow closed?

Thank you
Ben

Hi,

Kernow is merely a graphical interface for Saxon, you should post to that list:

https://lists.sourceforge.net/lists/listinfo/saxon-help

...or possibly to the more general 'xquery-talk', or the 'xsl-list' if
you want to go down the xslt streaming route.

If you post a small sample of your xml, along with the relevant part
of your query you should make some progress.

cheers
andrew

On 11 April 2012 10:13, Dr. Christian Dugast <du...@te...> wrote:
> Hi Andrew,
>
> Thanks for your precise help.
>
> I will look further into your proposition at rewriting xquery using less
> memory.
> But I am puzzled about it:
> The query is very simple ... it goes through a large number of nodes/pages
> (a file with 250MB has 44100 nodes) but each node/page covers in average
> 100KB (each node/page contains a max of 1000 elements).
>
> My understanding would be that after having worked on a node/page, Kernow
> should release memory related to that node/page: it seems to keep some
> information that grows with the number of nodes.
>
> So what is the information kept by Kernow and how to manage the garbage
> collector and tell him to release information?
>
> Thanks
>
> Christian
>
> -----Original Message-----
> From: Andrew Welch [mailto:and...@gm...]
> Sent: Tuesday, 10 April, 2012 3:51 PM
> To: du...@te...
> Cc: ker...@li...
> Subject: Re: [Kernow] xquery on large data: memory increase? or something
> else?
>
> Ok, that doesnt sound too far off.  You can read about the heap here:
>
> http://publib.boulder.ibm.com/infocenter/javasdk/tools/index.jsp?topic=%2Fco
> m.ibm.java.doc.igaa%2F_1vg000139b8b453-11951f1e7ff-8000_1001.html
>
> Some potential options for you at this point are:
>
> - use a 64bit operating system allowing you to allocate more memory
>
> - rewrite the query to use less memory, or not hold on to objects to allow
> the garbage collector to free up the memory.  If you choose this option, you
> can always ask for help on the xquery-talk mailing list.
>
> - switch to xslt and use streaming (requires commercial licence for Saxon)
>
> - stick with xquery use an XML database
>
> - write your own pre and post processor to break up the large file, process
> the chunks, then reassemble them
>
> cheers
> andrew
>
> On 10 April 2012 14:36, Dr. Christian Dugast <du...@te...> wrote:
>> Ok thanks.
>> The max number I can run seems to be 1300m, eventhough I have 4 GB of
>> RAM and the system caches uses about 1GB.
>>
>> Copying and pasting the cmd from the bat file directly into the cmd
>> line gives me the error message: " could not reserve enough space for
>> object heap. Could not create the JVM
>>
>> I have given the JVM the possibility to grow up to 3 GB  ... it does
>> not seem to be able to take them.
>>
>> Thanks for your help
>>
>> Christian
>>
>>
>> -----Original Message-----
>> From: Andrew Welch [mailto:and...@gm...]
>> Sent: Tuesday, 10 April, 2012 3:07 PM
>> To: du...@te...
>> Subject: Re: [Kernow] xquery on large data: memory increase? or
>> something else?
>>
>>> But it cannot deal with 250MB files.
>>> I have increased the value to 2048 ... and the bat file does not want
>>> to start or ... it opens the dos cmd window ... and disappears.
>>
>> If you open a new command windows (windows key -> r, type 'cmd', click
>> ok) then copy and paste the contents of the .bat into the new window
>> and run it, then it will still be available after kernow closes,
>> allowing you to see any messages.
>>
>>> I tried with 1536 (1048+512) but it does not work either ...
>>> What are the next possible numbers after 1024 that are valid?
>>> If I did write valid memory numbers, what is the next step?
>>
>> Any number is allowed, it doesn't have to be a multiple of anything.
>> You could try 1200, 1250 etc
>>
>> The next steps after this are to try the query outside of Kernow,
>> using Saxon directly:
>>
>> http://saxonica.com/documentation/using-xquery/commandline.xml
>>
>> If you still experience a problem there, you could try asking on the
>> saxon-help mailing list.
>>
>> It might be possible to rewrite your xquery to use less memory, or you
>> may need to look into using streaming (in xslt), however that requires
>> a commercial saxon license.
>>
>> Or of course get some more ram :)
>>
>>
>>
>> --
>> Andrew Welch
>> http://andrewjwelch.com
>>
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>



-- 
Andrew Welch
http://andrewjwelch.com

Ok, that doesnt sound too far off.  You can read about the heap here:

http://publib.boulder.ibm.com/infocenter/javasdk/tools/index.jsp?topic=%2Fcom.ibm.java.doc.igaa%2F_1vg000139b8b453-11951f1e7ff-8000_1001.html

Some potential options for you at this point are:

- use a 64bit operating system allowing you to allocate more memory

- rewrite the query to use less memory, or not hold on to objects to
allow the garbage collector to free up the memory.  If you choose this
option, you can always ask for help on the xquery-talk mailing list.

- switch to xslt and use streaming (requires commercial licence for Saxon)

- stick with xquery use an XML database

- write your own pre and post processor to break up the large file,
process the chunks, then reassemble them

cheers
andrew

On 10 April 2012 14:36, Dr. Christian Dugast <du...@te...> wrote:
> Ok thanks.
> The max number I can run seems to be 1300m, eventhough I have 4 GB of RAM
> and the system caches uses about 1GB.
>
> Copying and pasting the cmd from the bat file directly into the cmd line
> gives me the error message: " could not reserve enough space for object
> heap. Could not create the JVM
>
> I have given the JVM the possibility to grow up to 3 GB  ... it does not
> seem to be able to take them.
>
> Thanks for your help
>
> Christian
>
>
> -----Original Message-----
> From: Andrew Welch [mailto:and...@gm...]
> Sent: Tuesday, 10 April, 2012 3:07 PM
> To: du...@te...
> Subject: Re: [Kernow] xquery on large data: memory increase? or something
> else?
>
>> But it cannot deal with 250MB files.
>> I have increased the value to 2048 ... and the bat file does not want
>> to start or ... it opens the dos cmd window ... and disappears.
>
> If you open a new command windows (windows key -> r, type 'cmd', click
> ok) then copy and paste the contents of the .bat into the new window and run
> it, then it will still be available after kernow closes, allowing you to see
> any messages.
>
>> I tried with 1536 (1048+512) but it does not work either ...
>> What are the next possible numbers after 1024 that are valid?
>> If I did write valid memory numbers, what is the next step?
>
> Any number is allowed, it doesn't have to be a multiple of anything.
> You could try 1200, 1250 etc
>
> The next steps after this are to try the query outside of Kernow, using
> Saxon directly:
>
> http://saxonica.com/documentation/using-xquery/commandline.xml
>
> If you still experience a problem there, you could try asking on the
> saxon-help mailing list.
>
> It might be possible to rewrite your xquery to use less memory, or you may
> need to look into using streaming (in xslt), however that requires a
> commercial saxon license.
>
> Or of course get some more ram :)
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>



-- 
Andrew Welch
http://andrewjwelch.com

Hi,

Try changing the Xmx in the Kernow.bat file, then run Kernow using
that bat file.

cheers
andrew


On 10 April 2012 11:26, Dr. Christian Dugast <du...@te...> wrote:
> Hello,
>
>
>
> I want to run XQuery on medium to large files (from 100 MB to 600 MB of html
> code).
>
>
>
> If I run the query on a portion of these files that is smaller than 100 MB,
> everything runs well ...
>
> As soon as I want to run the query on the complete file, Kernow runs for a
> while and then it seems it does not run at all anymore, eventhough I do not
> get the hand back.
>
> CPU consumption goes from 30%-50% down to 1%-3%.
>
> Memory consumption went high ...from 700 MB up to 1,3 GB but I have plenty
> memory left.
>
>
>
> I have changed the JRE parameters to -Xmx1024m or even –Xmx2048m  (within
> the control panel /java/java/java runtime environment parameter settings)
>
> But it does not change anything ... I still have this magic limit of 100MB
> of data I can run an xQuery onto.
>
>
>
> What can I do? Where can I tell Kernow to effectively use more memory? Or is
> the problem another one?
>
>
>
> I run a windows XP SP3 PC with 4GB of physical mem
>
>
>
> Thanks
>
>
>
> Christian
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Better than sec? Nothing is better than sec when it comes to
> monitoring Big Data applications. Try Boundary one-second
> resolution app monitoring today. Free.
> http://p.sf.net/sfu/Boundary-dev2dev
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>



-- 
Andrew Welch
http://andrewjwelch.com

Hello,

 

I want to run XQuery on medium to large files (from 100 MB to 600 MB of html
code).

 

If I run the query on a portion of these files that is smaller than 100 MB,
everything runs well ... 

As soon as I want to run the query on the complete file, Kernow runs for a
while and then it seems it does not run at all anymore, eventhough I do not
get the hand back.

CPU consumption goes from 30%-50% down to 1%-3%.

Memory consumption went high ...from 700 MB up to 1,3 GB but I have plenty
memory left.

 

I have changed the JRE parameters to -Xmx1024m or even -Xmx2048m  (within
the control panel /java/java/java runtime environment parameter settings)

But it does not change anything ... I still have this magic limit of 100MB
of data I can run an xQuery onto.

 

What can I do? Where can I tell Kernow to effectively use more memory? Or is
the problem another one?

 

I run a windows XP SP3 PC with 4GB of physical mem

 

Thanks

 

Christian

 

 

There isn't a specific feature for that, but you could use a directory
transform with an identity stylesheet (or any stylesheet) and then if
the input isn't well formed the transform will fail.

On 16 March 2012 17:48, Mike Ferrando <ach...@gm...> wrote:
> Andrew,
> How can I get validation for "well-formed" only?
> -Tyichicus
>
> On Fri, Mar 16, 2012 at 12:56 PM, Andrew Welch <and...@gm...>
> wrote:
>>
>> Hi,
>>
>> You need to explain a little more about what you are trying to do -
>> you don't need to enter any namespaces for xsd validation (the
>> namespaces are specified in the xsds)
>>
>> cheers
>> andrew
>>
>> On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
>> > Friends,
>> > I was using Kernow with xml documents to validate them.
>> > They have namespaces, more than one.
>> > Do I need to enter more than one namespace into the namespace field
>> > under
>> > the validate tab?
>> > Thanks,
>> > -Tychicus
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > This SF email is sponsosred by:
>> > Try Windows Azure free for 90 days Click Here
>> > http://p.sf.net/sfu/sfd2d-msazure
>> > _______________________________________________
>> > Kernowforsaxon-help mailing list
>> > Ker...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>> >
>>
>>
>>
>> --
>> Andrew Welch
>> http://andrewjwelch.com
>>
>>
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Kernowforsaxon-help mailing list
>> Ker...@li...
>> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>



-- 
Andrew Welch
http://andrewjwelch.com

Andrew,
How can I get validation for "well-formed" only?
-Tyichicus

On Fri, Mar 16, 2012 at 12:56 PM, Andrew Welch <and...@gm...>wrote:

> Hi,
>
> You need to explain a little more about what you are trying to do -
> you don't need to enter any namespaces for xsd validation (the
> namespaces are specified in the xsds)
>
> cheers
> andrew
>
> On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
> > Friends,
> > I was using Kernow with xml documents to validate them.
> > They have namespaces, more than one.
> > Do I need to enter more than one namespace into the namespace field under
> > the validate tab?
> > Thanks,
> > -Tychicus
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > Kernowforsaxon-help mailing list
> > Ker...@li...
> > https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
> >
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>

Andrew,
I was clicking around and saw that I needed to put the ee license in the
root directory of the app.
Now it sees Saxon EE and validates perfectly.
Sorry to bother you,
-Tychicus

On Fri, Mar 16, 2012 at 12:56 PM, Andrew Welch <and...@gm...>wrote:

> Hi,
>
> You need to explain a little more about what you are trying to do -
> you don't need to enter any namespaces for xsd validation (the
> namespaces are specified in the xsds)
>
> cheers
> andrew
>
> On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
> > Friends,
> > I was using Kernow with xml documents to validate them.
> > They have namespaces, more than one.
> > Do I need to enter more than one namespace into the namespace field under
> > the validate tab?
> > Thanks,
> > -Tychicus
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > Kernowforsaxon-help mailing list
> > Ker...@li...
> > https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
> >
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>

Andrew,
Also, I have saxonee and a license to run it.
I put it into the lib directory, but the program does not seem to see it.
It is still grayed out in the options.
What do I need to do?
Thanks,
-Tychicus

On Fri, Mar 16, 2012 at 12:56 PM, Andrew Welch <and...@gm...>wrote:

> Hi,
>
> You need to explain a little more about what you are trying to do -
> you don't need to enter any namespaces for xsd validation (the
> namespaces are specified in the xsds)
>
> cheers
> andrew
>
> On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
> > Friends,
> > I was using Kernow with xml documents to validate them.
> > They have namespaces, more than one.
> > Do I need to enter more than one namespace into the namespace field under
> > the validate tab?
> > Thanks,
> > -Tychicus
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > Kernowforsaxon-help mailing list
> > Ker...@li...
> > https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
> >
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>

Andrew,
Good to read you.
Basically, I opened up this awesome program, and wanted to validate a
directory of xml files.
They have multiple namespaces.
There is a XML Schema text box on the validate tab.
Do I need to put any schema info into this?
This is an example of what I am trying to validate.
http://findingaids.loc.gov/mastermets/mss/2011/ms011001.xml
Thanks,
-Tychicus

On Fri, Mar 16, 2012 at 12:56 PM, Andrew Welch <and...@gm...>wrote:

> Hi,
>
> You need to explain a little more about what you are trying to do -
> you don't need to enter any namespaces for xsd validation (the
> namespaces are specified in the xsds)
>
> cheers
> andrew
>
> On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
> > Friends,
> > I was using Kernow with xml documents to validate them.
> > They have namespaces, more than one.
> > Do I need to enter more than one namespace into the namespace field under
> > the validate tab?
> > Thanks,
> > -Tychicus
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > Kernowforsaxon-help mailing list
> > Ker...@li...
> > https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
> >
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>

Hi,

You need to explain a little more about what you are trying to do -
you don't need to enter any namespaces for xsd validation (the
namespaces are specified in the xsds)

cheers
andrew

On 16 March 2012 16:49, Mike Ferrando <ach...@gm...> wrote:
> Friends,
> I was using Kernow with xml documents to validate them.
> They have namespaces, more than one.
> Do I need to enter more than one namespace into the namespace field under
> the validate tab?
> Thanks,
> -Tychicus
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Kernowforsaxon-help mailing list
> Ker...@li...
> https://lists.sourceforge.net/lists/listinfo/kernowforsaxon-help
>



-- 
Andrew Welch
http://andrewjwelch.com

Friends,
I was using Kernow with xml documents to validate them.
They have namespaces, more than one.
Do I need to enter more than one namespace into the namespace field under
the validate tab?
Thanks,
-Tychicus

Hello Christian Dugast,

When a browser is able to render a page nicely it does not mean the source code of that page is valid. Browsers are built to allow a lot of invalid or not even well-formed markup.

The errors you observe have nothing to do with the declared DOCTYPE, the source code is just not well-formed, it is not XML (even though it might be declared being XML). 

• A tag (like <meta />) has to be closed to be well-formed.
• Certain characters (&, <, ', ") have to be written as entities (&amp; &lt; &apos; &quot;) in certain locations.
• JavaScript code must be in a CDATA section to be treated as text by the XML parser

As long as your source document is not XML you are out of luck starting an XML-based process. 

For me, this is the beauty of the XML standard: It enforces the standard and therefore makes following processes reliable.

- Michael


Am 27.12.2011 um 22:16 schrieb Dr. Christian Dugast:

> Hello,
>  
> There is a public valid webpage from which I want to extract information using an XQuery.
> I have saved the code of this webpage in a txt file and alternatively as an html document.
> I have analysed the code and I have written accordingly my query to extract just the information I am looking for.
>  
> But before being able to make an Xquery, Kernow parses the  original code and comes with a series of error messages eventhough the code is valid as the webpage produces a nice page using my browser.
>  
> The errors I see show code written in HTML ... so It seems, the webpage I am looking at extracting information has been written in html *and* in Xhtml, but I am not sure this is the real problem.

[…]

> Below a  list of error messages I get with the related code that generates these errors (errors I get with either both headers, the original or the simplified one)
>  
> Error message:
> Line 15, Col 3 The element type "meta" must be terminated by the matching end-tag "</meta>".
> <meta name="robots" content="noindex,follow">        ß this is the one faulty, sure, but my browser has no problem with this code

[…]

> Error message:
> Line 39, Col 72 The reference to entity "f" must end with the ';' delimiter.
> <option  value="/recherche_antidot/recherche.php?s=&f=Vis&o=appreciation,DESC&acces_libre=1">Note (décroissante)</option>

[…]

> Error message:
> Line 721, Col 23 Element type "scr" must be followed by either attribute specifications, ">" or "/>".
> <script language="JavaScript" type="text/javascript">
>                                 document.write('<scr'+'ipt id="jspub99029" language="JavaScript"

--
_______________________________________________________________
Michael Müller-Hillebrand: Dokumentations-Technologie
Adobe Certified Expert, FrameMaker
Lösungen und Training, FrameScript, XML/XSL, Unicode
Blog: http://cap-studio.de/ - Tel. +49 (9131) 28747